Skip to content

DeepSeek R1 Distill Qwen 32B

Every measured variant of this model: how much quality each quant keeps, and the VRAM and speed it costs to run.

VRAM footprint vs Local Intelligence Index

LB-2026-07.2 | 25/22.5/22.5/22.5/7.5

This chart appears after the model's first measured run lands. Use the quant ladder below for file size and VRAM requirements.

Variant profiles

Complete rows are ordered by Local Intelligence Index; partial rows show their measured axes but are not ranked. The VRAM/Fits columns (8Kcontext) tell you what your card needs. Ranks are within this family's variants.

Column guide

Variant — quant label plus, where sha-verified, the publisher of the exact weights file benchmarked; rows without a source line predate artifact identity records.

VRAM @8k — model weights, KV cache, and runtime headroom at 8k context.

Fits — the smallest common GPU VRAM tier above that estimate.

Prefill tok/s — prompt-processing speed.

Decode tok/s — generated-token speed after the prompt.

Overall tok/s — completion throughput across the full benchmark.

File size — the benchmarked model artifact on disk.

Runtime — the serving engine and version.

Run — the immutable benchmark receipt. Live rows link to the public submission record until the run is baked into the static site.

Pick the largest quant whose VRAM @8k fits your card.

Swipe horizontally for all variant metrics →

Rank (this family)VariantLocal Intelligence Indexindex-v3.0 | 40/15/15/10/15/5VRAM @8kFitsOverall tok/sFile sizeRuntimeRun
—
Q8_0by unsloth
no run yet37.4 GB48 GB—34.8 GB—benchmark it
—
Q6_Kby unsloth
no run yet29.5 GB32 GB—26.9 GB—benchmark it
—
Q5_K_Mby unsloth
no run yet25.9 GB32 GB—23.3 GB—benchmark it
—
Q4_K_Mby unsloth
no run yet22.5 GB24 GB—19.9 GB—benchmark it
—
Q3_K_Mby unsloth
no run yet18.5 GB24 GB—15.9 GB—benchmark it
—
Q2_Kby unsloth
no run yet14.9 GB16 GB—12.3 GB—benchmark it