mannix/
llama-3.3

282 9 months ago

New state of the art 70B model. Llama 3.3 70B offers similar performance compared to Llama 3.1 405B model. I-Quants models.

tools

Models

View all →

9 models

llama-3.3:latest

40GB · 128K context window · Text · 9 months ago

llama-3.3:iq2_xxs

19GB · 128K context window · Text · 9 months ago

llama-3.3:iq2_xs

21GB · 128K context window · Text · 9 months ago

llama-3.3:iq2_s

22GB · 128K context window · Text · 9 months ago

llama-3.3:iq3_xxs

27GB · 128K context window · Text · 9 months ago

llama-3.3:iq3_xs

29GB · 128K context window · Text · 9 months ago

llama-3.3:iq3_s

31GB · 128K context window · Text · 9 months ago

llama-3.3:iq4_xs

38GB · 128K context window · Text · 9 months ago

llama-3.3:iq4_nl

40GB · 128K context window · Text · 9 months ago

Readme

  • Quantization from fp32
  • Using i-matrix calibration dataset calibration_datav3.txt
  • I-Quants models, only if passing tests iq2_xs, iq2_xxs, iq3_xxs, iq4_nl, iq3_s, iq2_s, iq4_xs, iq3_xs
  • Default quantization iq4_nl

image.png

New state of the art 70B model. Llama 3.3 70B offers similar performance compared to Llama 3.1 405B model.

Model reference