A fast inference library for running LLMs locally on modern consumer-class GPUs

Wheels for llama-cpp-python compiled with cuBLAS support

Fast and memory-efficient exact attention - Windows wheels

Python bindings for llama.cpp

AutoAWQ implements the AWQ algorithm for 4-bit quantization with a 2x speedup during inference. Documentation:

A combination of Oobabooga's fork and the main cuda branch of GPTQ-for-LLaMa in a package format.

An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.

8-bit CUDA functions for PyTorch

Windows compile of bitsandbytes for use in text-generation-webui.

LLM Frontend for Power Users.

An open source ChatGPT UI.

Merge Transformers language models by use of gradient parameters.

Robust Speech Recognition via Large-Scale Weak Supervision

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

Port of Facebook's LLaMA model in C/C++

4 bits quantization of LLaMa using GPTQ

one-click deepfake (face swap)

Precompiled Wheels for GPTQ-for-LLaMa