Allow for possibly non-pooled embeddings

[iamlemec]

Getting embeddings direct from "non-embedding" LLMs is pretty important for achieving SOTA performance (see MTEB Leaderboard). This PR makes it so that the embed functions return token level embeddings when the pooling_type ends up being LLAMA_POOLING_TYPE_NONE and the appropriate sequence level embeddings otherwise. After that, the user can do whatever type of pooling they wish. This should address most of the concerns raised in #1288 too.

There are still some gotchas. If you request mean/cls pooling on a model that doesn't support it, it will hard crash in llama.cpp. And it's hard to know ex ante what types of pooling a model supports. So it's probably just better to wait until this gets sorted out in llama.cpp.

Also, this changes the default for normalize to False. You typically want to normalize after pooling, so you would not want to normalize when getting token level embeddings. The other option is defaulting to False for token level and True for sequence level, but that seems overly complicated.

Note: this depends on the llama_pooling_type function, which just got merged in ggml-org/llama.cpp@b4e4b8a.

[abetlen]

Suggested change

	# LLAMA_API enum llama_pooling_type llama_pooling_type(const struct llama_model * model);
	@ctypes_function("llama_pooling_type", [llama_model_p_ctypes], ctypes.c_int)
	def llama_pooling_type(model: llama_model_p, /) -> int:
	...

I added this function elsewhere in the file to match the llama.h order so we can delete this.

[abetlen]

@iamlemec thank you so much for this!

I suppose expanding the possible return type of Embeddings is necessary to support sequences of embeddings and provide the ability to do your own pooling so I think that's okay.

Would you mind expanding the Embedding section in the README to explain this behaviour? It's quite bare at the moment and would be very helpful if anyone has questions here.

[iamlemec]

Yup, just updated the README!

[abetlen]

@iamlemec thank you that's fantastic! I'll go ahead and merge this now!