← Back to leaderboard  ·  Show all task instances

FFmpeg/FFmpeg

Mirror of https://git.ffmpeg.org/ffmpeg.git

59,217 c hard
3,050
Generated Behavioral Tests
5.3%
Best Score

Results by Model

# Model Score help_outline Percentage of hidden behavioral tests passed. Cost help_outline Total API cost in USD for this task instance. Calls help_outline Number of LLM API calls for this task instance.
1 Gemini 3.1 Pro Google 5.3% $0.92 52
2 GPT 5.4 OpenAI 4.6% $0.39 12
3 Claude Sonnet 4.6 Anthropic 4.5% $12.51 273
4 GPT 5.4 mini OpenAI 3.9% $0.03 13
5 Gemini 3 Flash Google 3.5% $0.38 89
6 Claude Opus 4.7 Anthropic 3.3% $1.07 35
7 Claude Haiku 4.5 Anthropic 2.8% $0.48 103
8 GPT 5 mini OpenAI 2.2% $0.02 15
9 Claude Opus 4.6 Anthropic 0.0% $2.06 221

Click row to see model details