krith/
llama-3.1-nemotron-70b-instruct-hf

229 1 year ago

I-quants for llama-3.1-nemotron-70b-instruct-hf

tools

Models

View all →

7 models

llama-3.1-nemotron-70b-instruct-hf:IQ3_M

32GB · 128K context window · Text · 1 year ago

llama-3.1-nemotron-70b-instruct-hf:IQ1_M

17GB · 128K context window · Text · 1 year ago

llama-3.1-nemotron-70b-instruct-hf:IQ2_XXS

19GB · 128K context window · Text · 1 year ago

llama-3.1-nemotron-70b-instruct-hf:IQ2_XS

21GB · 128K context window · Text · 1 year ago

llama-3.1-nemotron-70b-instruct-hf:IQ2_M

24GB · 128K context window · Text · 1 year ago

llama-3.1-nemotron-70b-instruct-hf:IQ3_XXS

27GB · 128K context window · Text · 1 year ago

llama-3.1-nemotron-70b-instruct-hf:IQ4_XS

38GB · 128K context window · Text · 1 year ago

Readme

Created using GGUF files from https://huggingface.co/bartowski/Llama-3.1-Nemotron-70B-Instruct-HF-GGUF