Reasoning Model Better <think> Integration

[RockasMockas]

Something that I think could be a general benefit for anyone using reasoning models (local, or via API) would be to have a config option that allows automatically omitting all content in <think> tags from being included in subsequent calls to the LLM in the same chat. Cuts down how much context is used and makes the more verbose models a bit better to work with.

Would you be interested in having this integrated into the plugin if I submitted a PR?

(Having a way to customize ... text visualization (ie. grey coloring) could also be nice, though based on the comments on the fidget integration I presume you're trying to be pretty stylistic-agnostic in general)

[olimorris]

automatically omitting all content in tags from being included in subsequent calls. Cuts down how much context is used and makes the more verbose models a bit better to work with.

I'm not sure why you'd want to do this. Most LLMs are stateless so by design so you need to pass in the entire conversation history with every response.

[RockasMockas]

If you're running LLMs locally your conversation history gets fatter and fatter, which depending on constraints of your ram/vram for context allowance, you end up being able to do a lot less with reasoning models over a lot of messages. Especially with models like qwq where the thinking ends up doing a few loops before deciding, there's limited value in including the thinking when the post thinking message holds vast majority of relevance for progressing forward on the task.

[RockasMockas]

Ah apologies, I realize I completely missed specifying that I meant <think> tags besides in the title, rather than tags in general. (the tag got eaten by markdown, fixed now)

[olimorris]

It's probably easier to just do gd and edit the chat buffer history followed by <c-s>

[RockasMockas]

Yeah but I instruct the LLM to do a job that takes say 15 tools calls, each with like 4-8 paragraphs of useless context that is irrelevant, it doesn't really scale to do manually and the point is to have code/LLMs automate such things in flows. If you don't want it in the plugin I can just hack something custom in my config to do it and it's fine either way at the end of the day.