Code

Fix adapter v2 llm.int8 inference#323

Code

Open

Diormiu wants to merge 8 commits intoLightning-AI:mainLightning-AI/lit-llama:mainfrom Diormiu:fix-adapterv2-int8Diormiu/lit-llama:fix-adapterv2-int8Copy head branch name to clipboard

Commits

Commits on May 24, 2023

• convert adapterv2 weight to input dtype

  Diormiu

  committed
  b77cbcc

  Copy full SHA for b77cbcc

Commits on May 25, 2023

• dequantize adapter/layers weights

  Diormiu

  committed
  2824293

  Copy full SHA for 2824293

Commits on May 26, 2023

• Update adapter_v2.py

  Show description for 2a57575
  rasbt

  authored
  2a57575

  Copy full SHA for 2a57575

Commits on Jun 2, 2023

• fix dtype usage

  rasbt

  committed
  961a522

  Copy full SHA for 961a522

• delete dataset symlink

  rasbt

  authored
  3d88c3c

  Copy full SHA for 3d88c3c

• make Linear8bitLt import optional

  rasbt

  authored
  ae36781

  Copy full SHA for ae36781

• another attempt

  rasbt

  authored
  257bcda

  Copy full SHA for 257bcda

Commits on Jun 7, 2023

• add model.reset_cache() back

  rasbt

  authored
  c93c39f

  Copy full SHA for c93c39f