Output in chat buffer not streaming

[fsiraj]

This started happening quite recently, but the output from the LLM no longer streams into the chat buffer - I have to wait for it all to show up at once.

I'm using the copilot adapter with the gpt4o model and have no complicated config. I've tried reverting to older versions of codecompanion but no luck. Any help in debugging this issue is appreciated.

I'm on neovim 0.11.

[olimorris]

Myself (and I assume many others) aren't experiencing this so I'd advise pairing back with the minimal.lua file in the repository. Are you changing the stream option in the adapter by chance?

[fsiraj]

I tried with the minimal.lua and same results. Not seeing any errors or anything in the logs. I double checked that gpt4o does support streaming. My own config for codecompanion is:

        opts = {
            display = {
                chat = { show_header_separator = false, auto_scroll = false },
            },
        },

I'm experiencing this on both MacOS and Ubuntu so I may just chalk it up to bad luck then.

Regardless, loving the plugin!

[olimorris]

If you've properly tried with the minimal.lua file then that means the issue is on OpenAI's side, which is unlikely.

Btw, auto_scroll will make it appear like data isn't being streamed because the cursor remains in the same place...

[fsiraj]

Here's what I get with minimal.lua. The auto_scroll = false behaves the same except the cursor doesn't end up at the bottom when the response pops up (as expected).

Screencast.from.2025-03-27.11-26-57.webm

I copied minimal.lua unchanged and ran nvim --clean -u minimal.lua

[olimorris]

So interestingly...that seem to be a gpt-4o issue - it doesn't support streaming. Try the gpt-4o-2024-11-20 model.

Edit - Seems an odd decision on GitHub's part

[fsiraj]

That fixed it! In case it helps anyone else:

        opts = {
            adapters = {
                copilot = function()
                    return require('codecompanion.adapters').extend('copilot', {
                        schema = { model = { default = 'gpt-4o-2024-11-20' } },
                    })
                end,
            }
        }

Just curious how did you check that the model doesn't support streaming? I incorrectly ruled that out initially after checking https://platform.openai.com/docs/models/gpt-4o where it lists all 3 versions as supporting streaming.

[olimorris]

I actually just switched to a few different models having been able to match your findings.