Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Metal installation documentation #1968

Open
devtobi opened this issue Mar 9, 2025 · 0 comments
Open

Metal installation documentation #1968

devtobi opened this issue Mar 9, 2025 · 0 comments

Comments

@devtobi
Copy link

devtobi commented Mar 9, 2025

I tried setting up llama-cpp-python in the current version 0.3.7 on my MacBook M4 Pro.
In the first step I only installed via pip install llama-cpp-python --no-cache-dir without specifiying the environment variable for Metal backend support.
I set the n_gpu_layers to -1 to fully use the GPU.

The interesting thing is the GPU was used even without having to install the Metal backend support as stated in the current documentation. I double checked this with a fresh start and explicitely setting the CMAKE_ARGS env variable and did not see and difference in terms of performance or GPU usage.

This is pretty handy, because when not using pip for dependency management (e.g. poetry) passing the environment variable did not work on my side.

Maybe the documentation should be updated to state that the env arguments are no longer required? This would also reflect the documentation in https://github.com/ggml-org/llama.cpp/blob/master/docs/build.md#metal-build where it states that: "On MacOS, Metal is enabled by default"

This would also mean, that custom pre-built wheels are no longer required as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant