Pinned Loading

1. zo2 zo2 Public

   ZO2 (Zeroth-Order Offloading): Full Parameter Fine-Tuning 175B LLMs with 18GB GPU Memory

   Python 95 7

2. Tiny-DeepSpeed Tiny-DeepSpeed Public

   Tiny-DeepSpeed, a minimalistic re-implementation of the DeepSpeed library

   Python 13 2

3. Flash-Attention-Implementation Flash-Attention-Implementation Public

   Implementation of Flash-Attention (both forward and backward) with PyTorch, CUDA, and Triton

   Python 2

4. Tiny-Megatron Tiny-Megatron Public

   Tiny-Megatron, a minimalistic re-implementation of the Megatron library

   Python 4

5. MetaProfiler MetaProfiler Public

   MetaProfiler is a lightweight, structure-agnostic operator-level profiler for PyTorch models that leverages MetaTensor execution to simulate and benchmark individual ops without loading the full mo…

   Python 1

6. simple_cuda_kernel simple_cuda_kernel Public

   A collection of ultra-simple yet high-performance CUDA kernels.

   Python 1