DARKEST UNIVERSE / I-MATRIX / 29B / I-QUANT

Huggingface user DavidAU has created some good MoE models of many sizes, focusing on prose, well made descriptions, and the horror genre. Notable models being the smaller “Hell California,” and the larger, non-MoE “Darkest Planet”. This non-MoE model is one of their larger models made. To stuff as many parameters in as little VRAM as possible, weighted K and I-quants will be listed.

Note that I-quants forfeit some token generation speed relative to K-quants in exchange for storage efficiency. The small 3-bit quants, I or K, are recommended for 16GB GPUs. These models were taken from GGUF formats from Huggingface.

Original model (DavidAU):

GGUF weighted quantizations (mradermacher):