142 1 year ago

Hermes 3 Llama-3.1 70b Model by NousResearch

tools

Models

View all →

22 models

hermes-3-llama-3.1-70b:latest

40GB · 128K context window · Text · 1 year ago

hermes-3-llama-3.1-70b:q2_k

26GB · 128K context window · Text · 1 year ago

hermes-3-llama-3.1-70b:q3_k_s

31GB · 128K context window · Text · 1 year ago

hermes-3-llama-3.1-70b:q3_k_m

34GB · 128K context window · Text · 1 year ago

hermes-3-llama-3.1-70b:q3_k_l

37GB · 128K context window · Text · 1 year ago

hermes-3-llama-3.1-70b:q4_0

40GB · 128K context window · Text · 1 year ago

hermes-3-llama-3.1-70b:q4_1

44GB · 128K context window · Text · 1 year ago

hermes-3-llama-3.1-70b:q4_k_s

40GB · 128K context window · Text · 1 year ago

hermes-3-llama-3.1-70b:q4_k_m

43GB · 128K context window · Text · 1 year ago

hermes-3-llama-3.1-70b:q5_1

53GB · 128K context window · Text · 1 year ago

hermes-3-llama-3.1-70b:q5_k_s

49GB · 128K context window · Text · 1 year ago

hermes-3-llama-3.1-70b:q5_k_m

50GB · 128K context window · Text · 1 year ago

hermes-3-llama-3.1-70b:q6_k

58GB · 128K context window · Text · 1 year ago

hermes-3-llama-3.1-70b:q8_0

75GB · 128K context window · Text · 1 year ago

hermes-3-llama-3.1-70b:iq2_xxs

19GB · 128K context window · Text · 1 year ago

hermes-3-llama-3.1-70b:iq2_xs

21GB · 128K context window · Text · 1 year ago

hermes-3-llama-3.1-70b:iq2_s

22GB · 128K context window · Text · 1 year ago

hermes-3-llama-3.1-70b:iq3_xxs

27GB · 128K context window · Text · 1 year ago

hermes-3-llama-3.1-70b:iq3_xs

29GB · 128K context window · Text · 1 year ago

hermes-3-llama-3.1-70b:iq3_s

31GB · 128K context window · Text · 1 year ago

hermes-3-llama-3.1-70b:iq4_xs

38GB · 128K context window · Text · 1 year ago

hermes-3-llama-3.1-70b:iq4_nl

40GB · 128K context window · Text · 1 year ago

Readme

  • Quantization from fp32
  • Using i-matrix calibration_datav3.txt
  • temperature set to 0.2: beware higher value will destroy reasoning and math capabilities of the model

Model Description

Hermes 3 is the latest version of our flagship Hermes series of LLMs by Nous Research.

For more details on new capabilities, training results, and more, see the Hermes 3 Technical Report.

Hermes 3 is a generalist language model with many improvements over Hermes 2, including advanced agentic capabilities, much better roleplaying, reasoning, multi-turn conversation, long context coherence, and improvements across the board.

The ethos of the Hermes series of models is focused on aligning LLMs to the user, with powerful steering capabilities and control given to the end user.

The Hermes 3 series builds and expands on the Hermes 2 set of capabilities, including more powerful and reliable function calling and structured output capabilities, generalist assistant capabilities, and improved code generation skills.

Benchmarks

Hermes 3 is competitive, if not superior, to Llama-3.1 Instruct models at general capabilities, with varying strengths and weaknesses attributable between the two.

Full benchmark comparisons below:

image.png