Intercept VS Code requests/responses

[S1M0N38]

Here is a simple way to debug/intercept Copilot requests in VS Code

1. Install mitm proxy
2. Install VS Code (Visual Studio Code - Insider contains the experimental/preview feature)
3. Install Copilot and Copilot Chat extensions
4. Sign in to GitHub Copilot in VS Code (make sure that Copilot features work)
5. Configure proxy in VS Code settings

This is equivalent to the following JSON config

{
    "http.proxy": "http://127.0.0.1:4141",
    "http.proxyStrictSSL": false
}

6. Start mitm proxy in a new terminal (the web interface is nice)

mitmweb --set listen_port=4141

7. Restart VS Code and inspect the intercepted requests/responses through the mitm web interface

---

Of course, you can do much more with mitm (e.g. custom scripts to dump req/res, patch req/res, ...)

[olimorris]

The amount of times in my life I could have done with this! Thanks for sharing this awesome tip

[S1M0N38]

When looking for model name, I've noticed that the /models endpoint returns the list of available models. Here is the response (Feb 25th 2025)

models.json

Moreover, I've noticed two changes in the requests:

1. The base URL for Copilot requests for individual plans is now https://api.individual.githubcopilot.com (but https://api.githubcopilot.com still works, also for individual plans)
2. OpenAI models are now called using their "simple id" (e.g. gpt-4o, gpt-4o-mini, o1, ...) and not with their complete name (e.g. gpt-4o-2024-08-06).

---

It's a bit of work, but if the /models endpoint is stable, the model choices for Copilot adapter can be dynamically generated from the response of this endpoint (this includes max_context_window_tokens, max_output_tokens, max_prompt_tokens).

[olimorris]

Amazing find! And you've answered an outstanding question I had regarding tool use via Copilot.

Any idea if GitHub have publicly documented that end point? I would imagine it's used by all of their editor native plugins so we could reliably use it.

Would you fancy adding a PR to support this?

[S1M0N38]

I don't think that copilot chat endpoints are publicly documented. I agree that they are using /models for list the available model (e.g. user has enable preview model in GitHub config) and to list the capabilities of the selected model (ctx window, tools, vision, ...). This is indeed one of the first call when opening vscode or vscode-copilot-chat.

I've to read a bit codecompanion.nvim codebase and understand how models scheme can be dinamically created. I'll give a shot this afternoon starting by creating models scheme with max supported tokens (window, input, output).

Do you think that this scheme should be create every time when user enter a chat with copilot adapter, or the schemes should be generated and add to codebase using CI (e.g. github action)?

[olimorris]

I've actually implemented something very similar with the Ollama adapter. It caches the output from their /v1/models endpoint (which is comparable to the Copilot endpoint) when an adapter is first called by the plugin. Ideally, I should put a time limit on that cache (in the way GitHub do with their Copilot tokens) to prevent the user from having to reload their Neovim instance. I think for now this approach is the way to go.

In the setup handler, we may need to check if each endpoint supports streaming, if not, we set it to false before making the request.

[S1M0N38]

I've dig deeper into the params used by requests performed by Copilot Chat Extension for VSCode.

Method

1. Open a new chat in the Copilot Chat panel (not the edit).
2. Select a model from the model picker.
3. Send the request "what can you do?"
4. Intercept the request with mitmweb.

I've collected the requests and saved them as .http (so I can easily modify and send them with rest.nvim)

• claude-3.5-sonnet.http.txt
• claude-3.7-sonnet-thought.http.txt
• claude-3.7-sonnet.http.txt
• gemini-2.0-flash-001.http.txt
• gpt-4o.http.txt
• o1.http.txt
• o3-mini.http.txt

(I had to add the .txt suffix in order to upload on this very github comment. Just remove it when download them)

The /models endpoint return this response

• models.json

Results

I'm interested in the relationship between models metadata's from /models and the params used in /chat/completions reqs.

chat/completions	default	modifiable
"model"	from /models "id"	yes
"temperature"	0.1	probably no
"top_p"	1	probably no
"max_tokens"	from /models "capabilities.limits.max_output_tokens"	yes, but auto-clip at default
"n"	1	no, it's always 1
"stream"	true if /models "capabilities.supports.streaming"	yes, can be always set to false

• All requests have all the parameters reported in the table.
• o1 is the only model from the picker that does not support streaming.
• in claude-3.5, a extra user message is injected to specifing the formatter (i.e. Use Markdown ...). Maybe to have a similar format to other models (gpt family) cuz Anthropic was/is pushing for xlm prompt format.
• The reasoning models do not show the reasoning tokens.
• The reasoning models do not use reasoning effort (so opts.can_reason is superfluous?).

I think that these results can be used for improving copilot schema. I can try to PR.
@olimorris

[olimorris]

Just reading this now. Surprised how similar the parameters are for all the models. Would welcome any feedback you have from testing and improving the schema. May also be worthwhile thinking about any underlying system prompt changes needed in the plugin too - although I've been reasonably happy with the base outputs for a while now.

[S1M0N38]

I've created llm-thermometer and have some evidence that temperature param is ignored:

• https://s1m0n38.github.io/llm-thermometer/reports/20250305T170658.html

(I've run the experiment just for o3-mini with few samples, cuz I don't want to be flag)

[olimorris]

Interesting and I think if I were to be asked this prior to seeing your results, I would have expected this. They've ultimately got a product to sell (at some point) and the risk of generating anything syntactically incorrect (which I would imagine increases with temperature) would be significant.

Love your research btw!

[S1M0N38]

Adding additional resources for intercepting req/res in text editor for future reference.

• https://github.com/LaiKash/cursor-aiserver-interceptor
• feat: mitm setup for #6 reachingforthejack/cursortab.nvim#12

[ravitemer]

@S1M0N38 Thanks. Do you have any experience with mitmproxy? I have tried to setup mitmproxy with setting proxy url and disabling ssl in VSCode. I can see some copilot completion requests at the start. But the Copilot Chat doesn't load and shows please connect to the internet.

[S1M0N38]

im not sure about that maybe try using local mode with mitmproxy

[olimorris]

@S1M0N38 - I'm wondering if you've come across how GitHub Copilot calls multiple tools? I've noticed that in #1141, Copilot stubbornly calls one tool at a time.

[S1M0N38]

I didn't take a close look at tool calls. By the UI, it seems that tools are called sequentially, but I'm not sure.

[olimorris]

I'll dig into this. Am keen to see if we can leverage GitHub's own tools as well.

[olimorris]

For macOS I had to add trust for mitmproxy's certificate:

- Open Keychain Access → System → Certificates → Import mitmproxy-ca-cert.pem → Set to "Always Trust".

and the certificate is located at ~/.mitmproxy/mitmproxy-ca-cert.pem

[S1M0N38]

Yep, that's step 4. of installation in this README.md