sqlite/sqlite | ProgramBench

Official Git mirror of the SQLite source tree

13,510

Generated Behavioral Tests

80.9%

Best Score

Muse Wonder (xhigh)

Results by Model

Hover for details · Click a model to open it · Drag to zoom the x-axis · Shift-drag to pan · The line marks the Pareto frontier (best score per cost)

30 runs

#	Model	Score	Cost	Calls
1	Muse Wonder (xhigh)	80.9%	$70.08	549	trace →
2	Muse Wonder Full	79.7%	TBA	532	trace →
3	Claude Opus 5.5 (xhigh)	77.0%	$50.50	296	trace →
4	Claude Fable 5.1 (xhigh)	74.2%	$74.14	234	trace →
5	Muse Spark 1.3 (max)	73.3%	$3.14	185	trace →
6	GPT 5.5 (xhigh)	71.6%	$14.62	155	trace →
7	Gemini 3.7 Flash	70.8%	$0.70	92	trace →
8	GPT 5.5 (high)	68.4%	$3.06	45	trace →
9	GLM-5.2	68.1%	$15.34	215	trace →
10	Gemini 3.6 Flash	67.9%	$3.58	156	trace →
11	Muse Spark 1.1 (xhigh)	67.6%	$0.84	169	trace →
12	Muse Spark 1.3 (xhigh)	67.0%	$1.20	212	trace →
13	Claude Opus 4.7	67.0%	$1.08	37	trace →
14	Muse Spark 1.2 (xhigh)	65.9%	$0.53	70	trace →
15	GPT-5.6 Sol	65.8%	$0.87	12	trace →
16	Claude Opus 4.7 (xhigh)	64.5%	$2.48	68	trace →
17	GPT 5.5	63.7%	$1.69	30	trace →
18	GPT 6 Astra (max)	1.8%	$14.48	54	trace →
19	Claude Sonnet 4.6	1.3%	$28.86	573	trace →
20	GPT-5.6 Sol (xhigh)	1.3%	$8.47	46	trace →
21	Gemini 3.5 Flash	1.0%	$6.54	277	trace →
22	Gemini 3 Flash	0.9%	$0.19	74	trace →
23	Claude Opus 4.8 (xhigh)	0.8%	$21.76	135	trace →
24	GPT 5.4	0.6%	$0.20	9	trace →
25	GPT 5.4 mini	0.5%	$0.02	11	trace →
26	Gemini 3.1 Pro	0.4%	$0.85	77	trace →
27	Claude Haiku 4.5	0.3%	$0.57	137	trace →
28	GPT 5 mini	0.2%	$0.01	14	trace →
29	Claude Opus 5 (xhigh)	0.0%	$7.13	70	trace →
30	Claude Opus 4.6	0.0%	$4.54	274	trace →

Click a row to replay how that model rebuilt this program, or the model name to open its full run