oppiliappan/statix | ProgramBench

lints and suggestions for the nix programming language

815

Generated Behavioral Tests

100.0%

Best Score

Claude Opus 5.5 (xhigh)

Results by Model

Hover for details · Click a model to open it · Drag to zoom the x-axis · Shift-drag to pan · The line marks the Pareto frontier (best score per cost)

30 runs

#	Model	Score	Cost	Calls
1	Claude Opus 5.5 (xhigh)	100.0%	$24.45	225	trace →
2	Muse Wonder Full	99.0%	TBA	375	trace →
3	Claude Fable 5.1 (xhigh)	98.9%	$69.24	318	trace →
4	Muse Wonder (xhigh)	98.5%	$29.58	272	trace →
5	GPT 6 Astra (max)	97.7%	$8.28	53	trace →
6	Claude Opus 5 (xhigh)	95.8%	$98.62	382	trace →
7	Muse Spark 1.3 (max)	92.1%	$4.33	309	trace →
8	GPT-5.6 Sol (xhigh)	89.8%	$11.43	71	trace →
9	GLM-5.2	85.1%	$35.48	205	trace →
10	Gemini 3.7 Flash	83.6%	$2.13	123	trace →
11	Muse Spark 1.3 (xhigh)	79.5%	$1.65	227	trace →
12	Claude Opus 4.8 (xhigh)	79.4%	$14.72	107	trace →
13	GPT 5.5 (xhigh)	69.0%	$9.28	83	trace →
14	GPT 5.5 (high)	65.2%	$4.29	48	trace →
15	GPT 5.5	62.1%	$1.19	18	trace →
16	GPT-5.6 Sol	59.4%	$1.03	15	trace →
17	Muse Spark 1.2 (xhigh)	58.9%	$1.06	127	trace →
18	Claude Opus 4.7	42.8%	$3.28	89	trace →
19	Claude Opus 4.7 (xhigh)	40.7%	$6.92	142	trace →
20	Gemini 3 Flash	30.6%	$0.20	51	trace →
21	Gemini 3.1 Pro	29.3%	$1.18	60	trace →
22	Gemini 3.5 Flash	29.2%	$6.32	164	trace →
23	GPT 5.4	25.4%	$0.30	11	trace →
24	GPT 5.4 mini	24.0%	$0.02	8	trace →
25	GPT 5 mini	21.7%	$0.01	18	trace →
26	Gemini 3.6 Flash	11.3%	$2.99	140	trace →
27	Muse Spark 1.1 (xhigh)	11.3%	$1.26	225	trace →
28	Claude Opus 4.6	11.3%	$14.17	282	trace →
29	Claude Sonnet 4.6	11.3%	$24.17	391	trace →
30	Claude Haiku 4.5	0.0%	$0.94	140	trace →

Click a row to replay how that model rebuilt this program, or the model name to open its full run