Cannot import Mistral7b

[juliusmlr]

On my Macbook Air m3 with MacOS 15.1 (24B83) i cannot import the mistral model downloaded from this repo: https://huggingface.co/apple/mistral-coreml

I'm running the preview branch.
This is the XCode error i'm running in:

.bundle/main/main_mps_graph/main_mps_graph.mpsgraphpackage/model_0.mpsgraph":0:0): error: attempting to parse a byte at the end of the bytecode
Models/LanguageModel.swift:39: Fatal error: Cannot obtain shape information

Any idea?

[rudrankriyam]

Facing the same issue

[kokluch]

same

[joaomelo81]

Switch the branch to main

[OneMHz]

If you look at model.modelDescription.inputDescriptionsByName, it contains key = inputIds. But the code is looking for input_ids. But if you fix that, then it complains about the causal mask because that's not added as an MLFeatureValue. And if you fix that, it complains that mistral needs an MLState (though, not in the most clear message), which is only iOS 18+/macOS15+. And if you fix that, it complains that the shape.count = 0 != stride.count = 3 or something (I had to update to the latest and need to re-apply some local changes to get the exact error back).

But I'm kind of naively fixing the error messages locally and hoping for the best so some guidance would be great.

[pcuenca]

Hi @OneMHz, are you using the preview branch?

[OneMHz]

No. Main. I did manage to code my way to getting a result. But it’s coming back as a MLMultiArray so that doesn’t work for the return type

[pcuenca]

You need the preview branch of this project, which also pulls the preview branch from swift-transformers. It adds supports for MLState and a few other things. Please, let us know if that works!

[pcuenca]

Sorry about that, we should unify soon!

[OneMHz]

Thanks! That is working much better!