← Back to leaderboard  ·  Show all task instances

nuta/nsh

A command-line shell like fish, but POSIX compatible.

966 rs medium
1,960
Generated Behavioral Tests
86.9%
Best Score
Muse Spark 1.3 (max)

Results by Model

Hover for details · Click a model to open it · Drag to zoom the x-axis · Shift-drag to pan · The line marks the Pareto frontier (best score per cost)

Per-Model Results

24 runs
# Model Score Cost Calls
1 Muse Spark 1.3 (max) 86.9% $4.63 240 trace →
2 Gemini 3.1 Pro 83.9% $0.92 64 trace →
3 GLM-5.2 83.6% $38.51 225 trace →
4 GPT-5.6 Sol (xhigh) 82.0% $7.07 47 trace →
5 Muse Spark 1.3 (xhigh) 81.3% $0.96 162 trace →
6 GPT 5.5 (xhigh) 81.1% $9.98 81 trace →
7 Claude Sonnet 4.6 80.6% $27.07 496 trace →
8 Claude Opus 4.8 (xhigh) 78.6% $27.98 158 trace →
9 Claude Opus 4.6 76.8% $13.22 304 trace →
10 GPT 5.5 (high) 76.3% $2.79 42 trace →
11 GPT 5 mini 76.2% $0.01 10 trace →
12 Claude Opus 4.7 (xhigh) 71.3% $5.86 116 trace →
13 Gemini 3.7 Flash 68.9% $1.80 130 trace →
14 Claude Opus 4.7 66.2% $3.70 92 trace →
15 Gemini 3.6 Flash 55.2% $7.28 204 trace →
16 GPT 5.5 53.0% $1.33 23 trace →
17 GPT 5.4 50.3% $0.19 8 trace →
18 Gemini 3.5 Flash 41.7% $7.17 192 trace →
19 Claude Haiku 4.5 36.8% $1.16 148 trace →
20 Gemini 3 Flash 31.3% $0.28 122 trace →
21 Muse Spark 1.2 (xhigh) 6.0% $2.78 206 trace →
22 Claude Opus 5 (xhigh) 0.0% $7.97 90 trace →
23 Muse Spark 1.1 (xhigh) 0.0% $0.66 194 trace →
24 GPT 5.4 mini 0.0% $0.02 14 trace →

Click a row to replay how that model rebuilt this program, or the model name to open its full run