This receipt lets anyone verify this run against the frozen suite — see Methodology.

suite-v2 | index-v4.2

Gemma 4 12B IT

gemma-4-12b-it__gemma-4-12b-it-qat-ud-q4kxl-s2v5

Local Intelligence Index

LB-2026-07.2 | 25/22.5/22.5/22.5/7.5

40.3

±2.7 95% CI

Profile: Agentic / Knowledge / Instruction / Coding / Math

Agentic 4.2 / Knowledge 75.6 / Instruction 66.7 / Coding 28.4 / Math 11.5

Weighted headline profile: Agentic 25%, Knowledge 22.5%, Instruction 22.5%, Coding 22.5%, Math 7.5%.

Total run time

17.2 h

Data quality note: this run has 0 error(s) and 103 no-answer item(s).

Data warnings

coding chance_corrected differs from item-derived mean; stale/inconsistent run JSON

Axis breakdown

Agenticworst axis

n=96 · errors=0 · no answer=0

4.2 ±5.8

Knowledge

n=400 · errors=0 · no answer=16

75.6 ±4.8

Instruction

n=294 · errors=0 · no answer=0

66.7 ±5.4

Coding

n=141 · errors=0 · no answer=0

28.4 ±7.8

Math

n=139 · errors=0 · no answer=87

11.5 ±5.8

Diagnostics · unweighted

Call formatting

n=330 · errors=0 · no answer=0

73.6 ±4.8

BFCL single-turn

— not measured

BFCL v3 multi-turn base — frozen snapshot

n=50 · errors=0 · no answer=6

24.0 ±12.0

BFCL multi-turn long-context

— not measured

RULER 32K

— not measured

Instruction-following decomposition

Strict = Termination × Conditional

Strict accuracy

66.7%

correct AND terminated / all

Termination rate

95.9%

terminated / all

Conditional accuracy

69.5%

correct / terminated

Outputs that hit the answer-token cap are counted incorrect; this prevents non-terminating generations from getting credit for matching required tokens inside a runaway response.

Manifest

model: gemma4
quant: QAT Q4_K_XL
runtime: llama.cpp b9852/fd1a05791 · KV k=f16,v=f16 · ctx 32,768
hardware: NVIDIA GeForce RTX 5090 (31.8 GB) · Windows-11-10.0.26200-SP0
os: Windows-11-10.0.26200-SP0
lane: bounded-final-v2
thinking_mode: n/a
caps: max_tokens_math: 0, max_tokens_mcq: 16384, thinking_budget: 8192
sampling: temp 0 | top_p n/a | top_k 1 | min_p n/a | seed 1234 | effort n/a | knowledge: max 16384 | instruction: max 16384 | coding: max 16384 | math: max 16384 | bfcl_multi_turn_base: max 16384 | tc_json_v1: max 16384
tokens: 8,355,858 prompt / 8,468,917 completion / 16,824,775 total
tokens-to-answer: 8,259 median / 16,384 p95
tok/s: 137.1
total run time: 17.2 h
est cost: n/a
n_items: 1,457
n_errors: 0
n_no_answer: 103

Serving performance

NVIDIA GeForce RTX 5090 (31.8 GB) · Windows-11-10.0.26200-SP0

prefill

5,383.5 tok/s

decode

147.5 tok/s

TTFT proxy

114.7 ms

prompt processing before first token — non-streaming harness, lower bound

coverage

93.4%

prompt median / p95

114.7 ms / 978.5 ms

predicted median / p95

54,977.8 ms / 109,719.3 ms

bench	prefill	decode	prompt median	n
amo	4,022.3 tok/s	149.6 tok/s	120.6 ms	39
bfcl_multi_turn_base	6,499.7 tok/s	128.8 tok/s	754.1 ms	50
bigcodebench_hard	6,073.7 tok/s	151.1 tok/s	103.7 ms	148
ifbench	5,203.1 tok/s	145.5 tok/s	106.2 ms	294
mmlu_pro	4,297.9 tok/s	150.6 tok/s	113.6 ms	400
olymmath_hard	2,349.8 tok/s	145.6 tok/s	117.1 ms	100
tc_json_v1	6,926.3 tok/s	152.5 tok/s	125.5 ms	330

Source: llama.cpp server timings.