Multiple model selection for different endpoints

[jukofyork]

I'm sorry to say it, but this just doesn't work:

• 95% of those models are no use for code.
• You get a 1-5s freeze every time you enter the settings page as it populates the list.
• You still have to type in a new Base Address and API Key when you want to switch endpoint.
• However hard I try to validate the URLs; it's still probably possible to put in an malformed URL that locks this page up and requires manual editing of the preference store datafile.

---

In actual use you need to:

1. Be able to easily switch between a handful of favourite models.
2. Each favourite model should be able to have duplicates with different temperature, nickname, etc.
3. A nice clean interface for adding, removing, sorting, editing, duplicating, and so on.
4. Some visual way of indicating if the model is online or not.

So I'm gonna have another go at this and give up on retrieving the model list for now.

[jukofyork]

So I've got a lot further and committed what I have done so far:

• It needs quite a bit of refactoring but not got time today.
• The 5 new buttons aren't respecting the Apply, Cancel, etc button for now.

• Bookmark: Copies the current settings into the bookmarks if not already in there.
• Unbookmark: Removes any that match the current settings.
• Clear: Just clears the bookmarks.
• Sort: Just sorts the bookmarks.
• Populate: Uses the current API URL and API Key settings, retrieves the model list, and then adds in any that match the current Model Name string.

Populate matches all models if the current Model Name string is blank:

Populate matches any models which have the current Model Name string as a sub-string otherwise:

Populate will copy the current setting of Temperature too:

---

I think this is the best compromise and avoids the problems of the page stalling each time you enter, etc.

It should hopefully be quite usable in practice, and allow say different copies of the models with Temperature being non-zero for discussion, etc.

[jukofyork]

I will have a look at this when I get more time next week:

Some visual way of indicating if the model is online or not.

[jukofyork]

The 5 new buttons aren't respecting the Apply, Cancel, etc button for now.

• This is fixed now so unless you press Apply the settings aren't saved to the preferences.
• Also fixed some funk where we were reading some stuff from the preference store instead of the FieldEditor widgets.

The HttpClientWrapper and OpenAiApiClient should be a lot harder against weird errors, and report back more clearly what is wrong.

Some visual way of indicating if the model is online or not.

This, and the ability to grey out buttons on the preference page for invalid model names/endpoints (in real time hopefully) is next.

[jukofyork]

the ability to grey out buttons on the preference page for invalid model names/endpoints (in real time hopefully) is next.

Done.

Some visual way of indicating if the model is online or not.

Added a validate button to check if the model name can be found at the given API endpoint:

Not sure if this will work in Ollama yet though, but it does work fine with OpenAI and Openrouter APIs.

This is all I've got time for now, so probably won't be pushing any more changes for a while until I see how it works in practice.

[jukofyork]

Not sure if this will work in Ollama yet though, but it does work fine with OpenAI and Openrouter APIs.

It should work with Ollama:

ollama/ollama@996bb1b

and llama.cpp too:

https://github.com/ggerganov/llama.cpp/blob/6423c65aa8be1b98f990cf207422505ac5a441a1/examples/server/server.cpp#L3129

and TabbyAPI:

https://github.com/theroyallab/tabbyAPI/blob/b8700fbbc3d08de24bffb32c62d24d8085482958/endpoints/core/router.py#L65

as well as these I've already tested:

https://platform.openai.com/docs/api-reference/models/list
https://openrouter.ai/docs/requests (https://openrouter.ai/api/v1/models)