Code

Add the RWKV model (v5).#1707

Code

Merged

LaurentMazare merged 12 commits intomainhuggingface/candle:mainfrom rwkvhuggingface/candle:rwkvCopy head branch name to clipboardFeb 14, 2024

+616-3

Commits

Commits on Feb 6, 2024

• Start adding the RWKV model.

  LaurentMazare

  committed
  8a5ba7d

  Copy full SHA for 8a5ba7d

• More of the forward step.

  LaurentMazare

  committed
  eae0fda

  Copy full SHA for eae0fda

• Handle rescaling.

  LaurentMazare

  committed
  ef82249

  Copy full SHA for ef82249

• FeedForward.

  LaurentMazare

  committed
  cb3ed50

  Copy full SHA for cb3ed50

Commits on Feb 13, 2024

• Merge remote-tracking branch 'origin/main' into rwkv

  LaurentMazare

  committed
  7b36645

  Copy full SHA for 7b36645

• More work on RWKV.

  LaurentMazare

  committed
  3c745e3

  Copy full SHA for 3c745e3

• Better state tracking.

  LaurentMazare

  committed
  1992aae

  Copy full SHA for 1992aae

• Finish a first pass on forward.

  LaurentMazare

  committed
  25c8bb3

  Copy full SHA for 25c8bb3

• Fix the shape mismatches.

  LaurentMazare

  committed
  0d6077c

  Copy full SHA for 0d6077c

Commits on Feb 14, 2024

• Do not rescale in f32.

  LaurentMazare

  committed
  b5dfe9d

  Copy full SHA for b5dfe9d

• Rename to rwkv-v5.

  LaurentMazare

  committed
  99ddf49

  Copy full SHA for 99ddf49

• Add the new models to the readme.

  LaurentMazare

  committed
  0a451b8

  Copy full SHA for 0a451b8