liangyuwang / Flash-Attention-Implementation Public

Notifications You must be signed in to change notification settings
Fork 0
Star 2

Implementation of Flash-Attention (both forward and backward) with PyTorch, CUDA, and Triton

Apache-2.0 license

2 stars 0 forks Branches Tags Activity

Notifications

Name		Name	Last commit message	Last commit date
Latest commit History 10 Commits
flashattn		flashattn
LICENSE		LICENSE
README.md		README.md

Repository files navigation

Flash-Attention-Implementation

Implementation of Flash-Attention (both forward and backward) with PyTorch, LibTorch, CUDA, and Triton

Geting Started

PyTorch

cd flashattn/pytorch
python flashattn.py

LibTorch
```
cd flashattn/libtorch
python test.py
```
CUDA
- TODO
Triton
- TODO

About

Implementation of Flash-Attention (both forward and backward) with PyTorch, CUDA, and Triton

Apache-2.0 license

Report repository

Releases

No releases published

Packages

No packages published

Languages