samtools/samtools | ProgramBench

Tools (written in C using htslib) for manipulating next-generation sequencing data

1,425

Generated Behavioral Tests

67.0%

Best Score

Claude Opus 5.5 (xhigh)

Results by Model

Hover for details · Click a model to open it · Drag to zoom the x-axis · Shift-drag to pan · The line marks the Pareto frontier (best score per cost)

30 runs

#	Model	Score	Cost	Calls
1	Claude Opus 5.5 (xhigh)	67.0%	$27.34	154	trace →
2	GPT 6 Astra (max)	64.6%	$28.08	95	trace →
3	Claude Fable 5.1 (xhigh)	60.8%	$84.79	286	trace →
4	Muse Wonder Full	56.9%	TBA	181	trace →
5	Muse Wonder (xhigh)	54.3%	$21.20	226	trace →
6	Claude Opus 5 (xhigh)	49.8%	$25.84	156	trace →
7	GPT-5.6 Sol (xhigh)	39.0%	$10.13	64	trace →
8	Muse Spark 1.3 (max)	32.9%	$5.51	294	trace →
9	Muse Spark 1.3 (xhigh)	25.6%	$1.27	179	trace →
10	GPT 5.5 (xhigh)	22.6%	$11.21	70	trace →
11	Gemini 3.7 Flash	22.3%	$1.03	90	trace →
12	GPT 5.5 (high)	21.5%	$5.82	35	trace →
13	GLM-5.2	21.3%	$12.22	113	trace →
14	GPT-5.6 Sol	18.8%	$1.37	15	trace →
15	Gemini 3.6 Flash	18.4%	$12.90	146	trace →
16	Claude Opus 4.8 (xhigh)	18.0%	$21.15	142	trace →
17	GPT 5.5	15.6%	$1.33	15	trace →
18	Claude Opus 4.6	14.2%	$10.17	192	trace →
19	GPT 5.4	7.1%	$0.33	10	trace →
20	Gemini 3.5 Flash	5.5%	$3.29	100	trace →
21	Gemini 3.1 Pro	5.2%	$1.43	99	trace →
22	Gemini 3 Flash	3.9%	$0.34	97	trace →
23	Claude Opus 4.7 (xhigh)	3.9%	$1.81	56	trace →
24	Muse Spark 1.1 (xhigh)	3.9%	$0.41	173	trace →
25	GPT 5.4 mini	3.9%	$0.04	8	trace →
26	Claude Opus 4.7	3.7%	$0.81	34	trace →
27	Claude Haiku 4.5	3.0%	$0.24	67	trace →
28	Muse Spark 1.2 (xhigh)	1.8%	$0.85	105	trace →
29	GPT 5 mini	1.8%	$0.02	16	trace →
30	Claude Sonnet 4.6	1.3%	$26.62	457	trace →

Click a row to replay how that model rebuilt this program, or the model name to open its full run