rs/curlie | ProgramBench

The power of curl, the ease of use of httpie.

693

Generated Behavioral Tests

99.6%

Best Score

Claude Opus 5.5 (xhigh)

Results by Model

Hover for details · Click a model to open it · Drag to zoom the x-axis · Shift-drag to pan · The line marks the Pareto frontier (best score per cost)

30 runs

#	Model	Score	Cost	Calls
1	Claude Opus 5.5 (xhigh)	99.6%	$10.39	169	trace →
2	Muse Wonder Full	98.8%	TBA	207	trace →
3	Muse Wonder (xhigh)	97.8%	$22.72	231	trace →
4	Muse Spark 1.3 (max)	97.1%	$3.34	247	trace →
5	Claude Fable 5.1 (xhigh)	96.7%	$20.74	147	trace →
6	Claude Opus 5 (xhigh)	95.8%	$18.53	146	trace →
7	Claude Opus 4.7 (xhigh)	92.9%	$18.14	157	trace →
8	Claude Opus 4.8 (xhigh)	92.9%	$14.69	109	trace →
9	GPT 5.5 (xhigh)	92.8%	$7.88	81	trace →
10	Gemini 3.7 Flash	92.5%	$1.84	142	trace →
11	Muse Spark 1.3 (xhigh)	91.2%	$1.97	258	trace →
12	Muse Spark 1.1 (xhigh)	89.9%	$0.52	240	trace →
13	Claude Opus 4.6	89.3%	$12.49	243	trace →
14	GLM-5.2	88.9%	$22.71	261	trace →
15	Claude Sonnet 4.6	88.3%	$25.25	518	trace →
16	GPT 5.5 (high)	87.7%	$3.25	38	trace →
17	GPT 6 Astra (max)	87.6%	$10.39	48	trace →
18	Muse Spark 1.2 (xhigh)	87.0%	$1.06	139	trace →
19	Gemini 3.6 Flash	85.1%	$4.78	122	trace →
20	Claude Opus 4.7	84.8%	$15.70	247	trace →
21	GPT 5.5	83.1%	$1.21	23	trace →
22	Gemini 3.5 Flash	80.4%	$2.41	99	trace →
23	Gemini 3.1 Pro	79.8%	$1.62	127	trace →
24	GPT-5.6 Sol (xhigh)	76.3%	$5.88	43	trace →
25	Gemini 3 Flash	71.1%	$0.22	85	trace →
26	GPT-5.6 Sol	70.0%	$1.17	19	trace →
27	GPT 5.4	54.0%	$0.27	10	trace →
28	Claude Haiku 4.5	43.0%	$0.59	118	trace →
29	GPT 5.4 mini	11.3%	$0.02	9	trace →
30	GPT 5 mini	0.0%	$0.03	16	trace →

Click a row to replay how that model rebuilt this program, or the model name to open its full run