Skip to content

0.13.0 🌋
Compare
Choose a tag to compare
@b4rtaz b4rtaz released this 23 Mar 22:11
· 5 commits to main since this release
v0.13.0
31ff8f4
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

This version introduces experimental GPU support using Vulkan. While Vulkan integration is still a work in progress - especially regarding shader performance - this marks the first step toward full GPU support.

How to build Distributed Llama with Vulkan support?

DLLAMA_VULKAN=1 make dllama

To run Distributed Llama with Vulkan please add the --gpu-index 0 argument. For example:

./dllama inference ... --gpu-index 0

Please ensure that the Vulkan SDK is installed on your machine. You can run the following command to check:

vulkaninfo
Assets 2
Loading