3,337 1 year ago

An open-source Mixture-of-Experts code language model that achieves performance comparable to GPT4-Turbo in code-specific tasks.

Models

View all →

23 models

deepseek-coder-v2-lite-instruct:latest

8.9GB · 4K context window · Text · 1 year ago

deepseek-coder-v2-lite-instruct:q2_k

6.4GB · 4K context window · Text · 1 year ago

deepseek-coder-v2-lite-instruct:q3_k_s

7.5GB · 4K context window · Text · 1 year ago

deepseek-coder-v2-lite-instruct:q3_k_m

8.1GB · 4K context window · Text · 1 year ago

deepseek-coder-v2-lite-instruct:q3_k_l

8.5GB · 4K context window · Text · 1 year ago

deepseek-coder-v2-lite-instruct:q4_0

8.9GB · 4K context window · Text · 1 year ago

deepseek-coder-v2-lite-instruct:q4_1

9.9GB · 4K context window · Text · 1 year ago

deepseek-coder-v2-lite-instruct:q4_k_s

9.5GB · 4K context window · Text · 1 year ago

deepseek-coder-v2-lite-instruct:q4_k_m

10GB · 4K context window · Text · 1 year ago

deepseek-coder-v2-lite-instruct:q5_0

11GB · 4K context window · Text · 1 year ago

deepseek-coder-v2-lite-instruct:q5_1

12GB · 4K context window · Text · 1 year ago

deepseek-coder-v2-lite-instruct:q5_k_s

11GB · 4K context window · Text · 1 year ago

deepseek-coder-v2-lite-instruct:q5_k_m

12GB · 4K context window · Text · 1 year ago

deepseek-coder-v2-lite-instruct:q6_k

14GB · 4K context window · Text · 1 year ago

deepseek-coder-v2-lite-instruct:q8_0

17GB · 4K context window · Text · 1 year ago

deepseek-coder-v2-lite-instruct:iq2_xs

6.0GB · 4K context window · Text · 1 year ago

deepseek-coder-v2-lite-instruct:iq2_s

6.0GB · 4K context window · Text · 1 year ago

deepseek-coder-v2-lite-instruct:iq3_xxs

7.0GB · 4K context window · Text · 1 year ago

deepseek-coder-v2-lite-instruct:iq3_xs

7.1GB · 4K context window · Text · 1 year ago

deepseek-coder-v2-lite-instruct:iq3_s

7.5GB · 4K context window · Text · 1 year ago

deepseek-coder-v2-lite-instruct:iq4_xs

8.6GB · 4K context window · Text · 1 year ago

deepseek-coder-v2-lite-instruct:iq4_nl

8.9GB · 4K context window · Text · 1 year ago

deepseek-coder-v2-lite-instruct:fp16

31GB · 4K context window · Text · 1 year ago

Readme

  • Quantization from fp32
  • Using i-matrix calibration_datav3.txt
  • New template:
    • should work with flash_attention
    • doesn’t forget the SYSTEM prompt
    • doesn’t forget the context
  • N.B: if the output breaks ask for repeat (but it shouldn’t with these quants)

DeepSeek-Coder-V2 is an open-source Mixture-of-Experts (MoE) code language model that achieves performance comparable to GPT4-Turbo in code-specific tasks. DeepSeek-Coder-V2 is further pre-trained from DeepSeek-Coder-V2-Base with 6 trillion tokens sourced from a high-quality and multi-source corpus.

Maximum context length: 128K - q4_0 on a RTX3090 24GB can fit in VRAM up to 46K context

References

Hugging Face