Paper: Where the Bits Go (Sun, 2026) · Recipe and data: GEHIRNS/magi · Website: gehirns.pages.dev
Capability per GiB. We study how much a language model can do inside a single 12 GB consumer GPU. Every model is scored on a fixed 12-suite benchmark (coding, reasoning, knowledge, tool calls) with thinking on, and the score is set against the GPU memory the model actually occupies at a 32K context. All numbers come from our own runs on one RTX 5070.
magi-1.0
GEHIRNS/magi-1.0 is Qwen3.5-4B in 1.80 GiB, with no training. Each tensor class is stored at the precision its measured sensitivity calls for, using only standard GGUF types, so the file runs in stock llama.cpp and Ollama. With an 8-bit KV cache (magi-1.0 + q8 KV) it keeps 96% of the full model's benchmark score in 30% of the GPU memory and decodes 2.7× faster.
| Baselines | Ours | ||||
|---|---|---|---|---|---|
| Benchmark | BF16 8.98 GiB | Q4_K_M 3.78 GiB | Q2_K 3.15 GiB | magi-1.0 3.02 GiB | magi-1.0 + q8 KV 2.65 GiB |
| Efficiency | |||||
| GPU memory at 32K (GiB) | 8.98 | 3.78 | 3.15 | 3.023.0× less | 2.653.4× less |
| Benchmark score | 83.96% | 84.14% | 77.06% | 81.30%97% | 80.64%96% |
| Capability density | 0.294 | 0.703 | 0.675 | 0.8002.7× | 0.8933.0× |
| Decode speed (tok/s) | 66.3 | 146.6 | 165.5 | 180.02.7× | 181.42.7× |
The benchmark score is the mean pass rate over the 12 suites. Capability density ρ is −log₂(1 − score) per GiB of GPU memory, that is, how many times the error rate halves per gibibyte. Small numbers are relative to BF16. Q4_K_M is the more accurate choice; magi-1.0 is the choice when memory is the constraint.
Models
| Model | What it is | Status |
|---|---|---|
| magi-1.0 | Qwen3.5-4B, sensitivity-allocated mixed precision, no training | recommended |
| magi-beta-1 | magi-1.0 with the full-attention k, v and output projections at Q6_K | for comparison |
| magi-alpha-1, magi-alpha-2 | Qwen3.5-9B with supervised fine-tuning | retired, not recommended |
The alphas are an earlier, separate experiment. The fine-tuning did not improve the base model's coding (LiveCodeBench 49 and 48 of 150, against 50 for the untrained base, within run-to-run variation), and magi-alpha-1 can degenerate in chat. They stay online for transparency.
The paper Where the Bits Go: Sensitivity-Allocated Mixed Precision for a 4B Reasoning Model on a 12 GB Consumer GPU (Eason Sun, 2026; DOI: 10.5281/zenodo.23630199) describes the method and all measurements. The build recipe, the importance matrix and the data behind every table and figure are at GEHIRNS/magi on GitHub.