← Back to detailed results

Claude Opus 4.8 (xhigh)

Anthropic · mini-SWE-agent · Rank #4

0%
Resolved help_outline Percentage of instances fully solved as measured by hidden behavioral tests.
16.5%
Almost Resolved help_outline Instances where the agent's solution passes ≥ 95% of all hidden behavioral tests.
$4205
Total Cost help_outline Total API cost in USD across all task instances.
28,977
Total Calls help_outline Total number of LLM calls across all task instances.

Score (Behavioral Test Pass Rate) Distribution

Per-Instance Results

200 instances
# Repository Lang Score Cost Calls
1 agourlay/zip-password-finder Find the password of protected ZIP files. rs 99.9% $11.27 111
2 tomarrell/wrapcheck A Go linter to check that errors from external packages are wrapped go 99.8% $24.77 180
3 wintermute-cell/ngrrram A TUI tool to help you type faster and learn new layouts. Includes a free cat. rs 99.7% $21.03 137
4 ekzhang/bore ๐Ÿ•ณ bore is a simple CLI tool for making tunnels to localhost rs 99.5% $10.72 102
5 wfxr/csview ๐Ÿ“  Pretty and fast csv viewer for cli with cjk/emoji support. rs 99.4% $14.02 129
6 wfxr/code-minimap ๐Ÿ›ฐ A high performance code minimap render. rs 99.4% $8.64 77
7 clog-tool/clog-cli Generate beautiful changelogs from your Git commit history rs 99.1% $10.87 122
8 Miserlou/Loop UNIX's missing `loop` command rs 98.9% $18.57 135
9 oppiliappan/eva a calculator REPL, similar to bc(1) rs 98.7% $17.63 119
10 shashwatah/jot โšกRapid note management for the terminal. rs 98.7% $28.13 195
11 htop-dev/htop htop - an interactive process viewer c 98.5% $10.49 93
12 sitkevij/hex ๐Ÿ”ฎ Futuristic take on hexdump, made in Rust. rs 98.5% $12.02 94
13 Byron/dua-cli View disk space usage and delete unwanted data, fast. rs 98.3% $32.06 200
14 rbakbashev/elfcat ELF visualizer. Generates HTML files from ELF binaries. rs 98.2% $15.56 124
15 abishekvashok/cmatrix Terminal based "The Matrix" like implementation c 98.2% $6.91 76
16 BLAKE3-team/BLAKE3 the official Rust and C implementations of the BLAKE3 cryptographic hash function rs 98.1% $8.26 93
17 jarun/nnn nยณ The unorthodox terminal file manager c 98.1% $15.25 151
18 pemistahl/grex A command-line tool and Rust library with Python bindings for generating regular expressions from user-provided test cases rs 98.1% $17.62 125
19 sheepla/pingu ๐Ÿงping command but with pingu go 97.6% $8.15 77
20 yaa110/nomino Batch rename utility for developers rs 97.4% $15.84 134
21 tarka/xcp An extended `cp` rs 96.9% $8.45 87
22 ismaelgv/rnr A command-line tool to batch rename files and directories rs 96.7% $18.82 133
23 anordal/shellharden The corrective bash syntax highlighter rs 96.7% $63.32 271
24 mgdm/htmlq Like jq, but for HTML. rs 96.6% $16.43 115
25 facebookresearch/fastText Library for fast text representation and classification. cpp 96.5% $10.96 79
26 xorg62/tty-clock Clock using lib ncurses c 96.4% $10.04 82
27 chmln/sd Intuitive find & replace CLI (sed alternative) rs 96.4% $10.85 102
28 ogham/dog A command-line DNS client. rs 96.3% $17.64 145
29 simeg/eureka ๐Ÿ’ก CLI tool to input and store your ideas without leaving the terminal rs 96.2% $10.74 77
30 sstadick/hck A sharp cut(1) clone. rs 96.0% $25.00 151
31 sharkdp/hexyl A command-line hex viewer rs 95.7% $24.50 157
32 riquito/tuc When cut doesn't cut it rs 95.4% $26.21 143
33 jrnxf/thokr โœจ sleek typing tui with visualized results and historical logging rs 95.2% $14.61 105
34 brocode/fblog Small command-line JSON Log viewer rs 95.0% $13.69 130
35 orf/gping Ping, but with a graph rs 95.0% $20.74 134
36 hooklift/gowsdl WSDL2Go code generation as well as its SOAP proxy go 94.9% $40.33 233
37 git-bahn/git-graph Command line tool to show clear git graphs arranged for your branching model rs 94.7% $43.65 230
38 sharkdp/pastel A command-line tool to generate, analyze, convert and manipulate colors rs 94.7% $30.75 181
39 rs/jplot iTerm2 expvar/JSON monitoring tool go 94.7% $25.17 129
40 trasta298/keifu Git genealogy, untangled. A TUI for navigating commit graphs with color and clarity. rs 94.7% $30.71 173
41 rust-ethereum/ethabi Encode and decode smart contract invocations rs 94.5% $24.18 175
42 nachoparker/dutree a tool to analyze file system usage written in Rust rs 94.5% $19.22 112
43 svenstaro/miniserve ๐ŸŒŸ For when you really just want to serve some files over HTTP right now! rs 94.2% $44.46 251
44 astro/deadnix Scan Nix files for dead code rs 94.2% $23.32 118
45 chmln/handlr A better xdg-utils rs 94.2% $20.07 154
46 BurntSushi/xsv A fast CSV command line toolkit written in Rust. rs 94.1% $23.30 156
47 sharkdp/fd A simple, fast and user-friendly alternative to 'find' rs 93.9% $17.92 145
48 rhysd/kiro-editor A small terminal UTF-8 text editor written in Rust ๐Ÿ“๐Ÿฆ€ rs 93.4% $49.47 217
49 pier-cli/pier A CLI to organize and run short Unix shell scripts rs 93.2% $13.72 117
50 cordx56/rustowl Visualize Ownership and Lifetimes in Rust rs 93.0% $8.41 91
51 guumaster/hostctl Your dev tool to manage /etc/hosts like a pro! go 93.0% $18.99 163
52 rs/curlie The power of curl, the ease of use of httpie. go 92.9% $14.69 109
53 konradsz/igrep Interactive Grep rs 92.9% $32.24 202
54 eudoxia0/hashcards A plain text-based spaced repetition system. rs 92.6% $17.98 145
55 mookid/diffr Yet another diff highlighting tool rs 91.7% $26.68 134
56 eradman/entr Run arbitrary commands when files change c 91.6% $10.41 126
57 ninja-build/ninja a small build system with a focus on speed cpp 90.5% $21.96 135
58 yoav-lavi/melody Melody is a language that compiles to regular expressions and aims to be more readable and maintainable rs 90.5% $31.56 154
59 kyoh86/richgo Enrich `go test` outputs with text decorations. go 90.3% $10.40 71
60 kyoheiu/felix tui file manager with vim-like key mapping rs 90.2% $11.03 121
61 mibk/dupl a tool for code clone detection go 89.5% $9.35 69
62 dalance/amber A code search / replace tool rs 89.4% $28.09 164
63 hush-shell/hush Hush is a unix shell based on the Lua programming language rs 89.3% $51.55 301
64 WGUNDERWOOD/tex-fmt An extremely fast LaTeX formatter written in Rust rs 89.0% $26.19 174
65 noborus/ov ๐ŸŽ‘Feature-rich terminal-based text viewer. It is a so-called terminal pager. go 88.8% $9.40 102
66 tomnomnom/gron Make JSON greppable! go 88.8% $8.86 84
67 blacknon/hwatch A modern alternative to the watch command, records the differences in execution results and can check this differences at after. rs 88.7% $16.75 139
68 altdesktop/i3-style ๐ŸŽจ Make your i3 config a little more stylish. rs 88.7% $28.39 141
69 raviqqe/muffet Fast website link checker in Go go 87.6% $17.85 158
70 psampaz/go-mod-outdated Find outdated dependencies of your Go projects. go-mod-outdated provides a table view of the go list -u -m -json all command which lists all dependencies of a Go project and their available minor and patch updates. It also provides a way to filter indirect dependencies and dependencies without updates. go 87.4% $5.31 63
71 XAMPPRocky/tokei Count your code, quickly. rs 86.9% $25.80 160
72 pls-rs/pls pls is a prettier and powerful ls(1) for the pros. rs 86.7% $48.46 261
73 elkowar/pipr A tool to interactively write shell pipelines. rs 86.6% $24.83 193
74 incu6us/goimports-reviser Right imports sorting & code formatting tool (goimports alternative) go 86.6% $22.84 194
75 Esubaalew/run Universal multi-language runner and smart REPL written in Rust. rs 86.5% $14.03 122
76 segmentio/chamber CLI for managing secrets go 86.4% $12.86 128
77 jonas/tig Text-mode interface for git c 85.5% $15.90 163
78 TheZoraiz/ascii-image-converter A cross-platform command-line tool to convert images into ascii art and print them on the console. Now supports braille art! go 85.4% $34.95 188
79 ajeetdsouza/zoxide A smarter cd command. Supports all major shells. rs 85.1% $31.19 170
80 Y2Z/monolith โฌ›๏ธ CLI tool and library for saving complete web pages as a single HTML file rs 85.0% $31.97 190
81 junegunn/fzf :cherry_blossom: A command-line fuzzy finder go 84.9% $27.12 139
82 lz4/lz4 Extremely Fast Compression algorithm c 84.5% $18.31 120
83 bensadeh/tailspin ๐ŸŒ€ A log file highlighter rs 84.5% $24.91 150
84 HaliteChallenge/Halite @twosigma's first artificial intelligence programming challenge cpp 84.4% $14.45 106
85 nikolassv/bartib A simple timetracker for the command line. It saves a log of all tracked activities as a plaintext file and allows you to create flexible reports. rs 84.3% $33.86 204
86 noborus/trdsql CLI tool that can execute SQL queries on CSV, LTSV, JSON, YAML and TBLN. Can output to various formats. go 84.1% $15.68 136
87 JohannesKaufmann/html-to-markdown โš™๏ธ Convert HTML to Markdown. Even works with entire websites and can be extended through rules. go 84.0% $13.28 140
88 codesnap-rs/codesnap ๐Ÿฆ€๏ธ๐Ÿ“ธ Pure Rust tool to generate beautiful code snapshots, provide CLI and Library rs 83.3% $14.74 135
89 cweill/gotests Automatically generate Go test boilerplate from your source code. go 83.3% $30.39 189
90 ArthurSonzogni/json-tui A JSON terminal UI made in C++ cpp 83.0% $17.18 102
91 peco/peco Simplistic interactive filtering tool go 82.8% $41.44 230
92 naggie/dstask Git powered terminal-based todo/note manager -- markdown note page per task. Single binary! go 82.5% $20.98 169
93 astaxie/bat Go implement CLI, cURL-like tool for humans go 82.4% $14.77 111
94 quinn-rs/quinn Async-friendly QUIC implementation in Rust rs 82.2% $9.21 86
95 bootandy/dust A more intuitive version of du in rust rs 81.8% $53.90 272
96 kisielk/errcheck errcheck checks that you checked errors. go 81.8% $12.79 111
97 Canop/rhit A nginx log explorer rs 81.8% $22.51 141
98 lua/lua A copy of the Lua development repository, as seen by the Lua team. Mirrored irregularly. All communication should be through the Lua mailing list https://www.lua.org/lua-l.html c 81.5% $23.23 148
99 Lymphatus/caesium-clt Caesium Command Line Tools - Lossy/lossless image compression tool rs 80.9% $9.88 85
100 sibprogrammer/xq Command-line XML and HTML beautifier and content extractor go 80.5% $17.74 119
101 oppiliappan/statix lints and suggestions for the nix programming language rs 79.4% $14.72 107
102 lh3/seqtk Toolkit for processing sequences in FASTA/Q formats c 79.3% $12.27 95
103 hairyhenderson/gomplate A flexible commandline tool for template rendering. Supports lots of local and remote datasources. go 79.2% $26.15 183
104 ggreer/the_silver_searcher A code-searching tool similar to ack, but faster. c 79.1% $11.74 95
105 o2sh/onefetch Command-line Git information tool rs 79.1% $21.12 145
106 lfos/calcurse A text-based calendar and scheduling application c 78.7% $25.73 175
107 nuta/nsh A command-line shell like fish, but POSIX compatible. rs 78.6% $27.98 158
108 antonmedv/walk Terminal file manager go 78.5% $32.54 174
109 alexpovel/srgn A grep-like tool which understands source code syntax and allows for manipulation in addition to search rs 78.4% $13.93 106
110 sclevine/yj CLI - Convert between YAML, TOML, JSON, and HCL. Preserves map order. go 78.3% $12.11 104
111 hatoo/oha Ohayou(ใŠใฏใ‚ˆใ†), HTTP load generator, inspired by rakyll/hey with tui animation. rs 78.3% $25.11 174
112 antonmedv/fx Terminal JSON viewer & processor go 78.2% $26.72 182
113 cmatsuoka/figlet Claudio's FIGlet tree c 78.2% $15.15 106
114 BurntSushi/ripgrep ripgrep recursively searches directories for a regex pattern while respecting your gitignore rs 78.0% $23.89 158
115 rcoh/angle-grinder Slice and dice logs on the command line rs 77.7% $19.74 148
116 sayanarijit/xplr A hackable, minimal, fast TUI file explorer rs 77.1% $29.21 206
117 Isona/dirble Fast directory scanning and scraping tool rs 76.8% $24.57 174
118 YS-L/flamelens Flamegraph viewer in the terminal rs 76.7% $20.34 157
119 foriequal0/git-trim Automatically trims your branches whose tracking remote refs are merged or stray rs 76.6% $25.96 184
120 direnv/direnv unclutter your .profile go 76.5% $31.73 215
121 FiloSottile/age A simple, modern and secure encryption tool (and Go library) with small explicit keys, no config options, and UNIX-style composability. go 76.0% $27.10 158
122 zevv/duc Dude, where are my bytes: Duc, a library and suite of tools for inspecting disk usage c 75.4% $32.62 171
123 cslarsen/jp2a Converts jpg images to ASCII c 75.0% $31.35 156
124 gabotechs/dep-tree Tool for helping developers keep their code bases clean and decoupled. It allows visualising a code base complexity using a 3d force-directed graph of files and the dependencies between them. go 74.5% $17.28 163
125 mkj/dropbear Dropbear SSH c 74.4% $7.67 91
126 Drew-Alleman/DataSurgeon Quickly Extracts IP's, Email Addresses, Hashes, Files, Credit Cards, Social Security Numbers and a lot More From Text rs 74.3% $22.43 146
127 mfridman/tparse CLI tool for summarizing go test output. Pipe friendly. CI/CD friendly. go 74.1% $15.57 147
128 LuaJIT/LuaJIT Mirror of the LuaJIT git repository c 73.9% $23.18 138
129 eliukblau/pixterm Draw images in your ANSI terminal with true color go 73.5% $30.02 136
130 kaushiksrini/parqeye Peek inside Parquet files right from your terminal rs 73.4% $23.62 150
131 unhappychoice/gittype A CLI code-typing game that turns your source code into typing challenges rs 73.3% $11.52 136
132 svenstaro/genact ๐ŸŒ€ A nonsense activity generator rs 73.2% $23.01 185
133 jhspetersson/fselect Find files with SQL-like queries rs 72.6% $22.17 208
134 multiprocessio/dsq Commandline tool for running SQL queries against JSON, CSV, Excel, Parquet, and more. go 72.5% $11.09 104
135 AmmarAbouZor/tui-journal Your journal app if you live in a terminal rs 72.3% $12.96 132
136 rust-embedded/svd2rust Generate Rust register maps (`struct`s) from SVD files rs 72.3% $17.07 137
137 skeema/skeema Declarative pure-SQL schema management for MySQL and MariaDB go 72.1% $9.97 103
138 cheat/cheat cheat allows you to create and view interactive cheatsheets on the command-line. It was designed to help remind *nix system administrators of options for commands that they use frequently, but not frequently enough to remember. go 70.4% $16.65 145
139 dundee/gdu Fast disk usage analyzer with console interface written in Go go 70.3% $13.86 107
140 yassinebridi/serpl A simple terminal UI for search and replace, ala VS Code. rs 69.5% $20.65 180
141 robertdavidgraham/masscan TCP port scanner, spews SYN packets asynchronously, scanning entire Internet in under 5 minutes. c 67.4% $12.75 150
142 sirwart/ripsecrets A command-line tool to prevent committing secret keys into your source code rs 66.6% $22.96 141
143 Canop/broot A new way to see and navigate directory trees : https://dystroy.org/broot rs 65.6% $13.01 110
144 sharkdp/bat A cat(1) clone with wings. rs 63.8% $26.98 170
145 rust-lang/mdBook Create book from markdown files. Like Gitbook but implemented in Rust rs 63.6% $24.82 184
146 NikolaDucak/caps-log A small TUI journaling tool. ๐Ÿ“– cpp 62.4% $12.66 112
147 facebook/zstd Zstandard - Fast real-time compression algorithm c 60.6% $25.52 143
148 ecumene/rust-sloth A 3D software rasterizer... for the terminal! rs 60.1% $23.02 135
149 zk-org/zk Plain text note-taking assistant go 59.7% $20.06 163
150 ariga/atlas Declarative schema migrations with schema-as-code workflows go 57.6% $8.92 95
151 Epistates/treemd A (TUI/CLI) markdown navigator with tree-based structural navigation. rs 57.4% $20.09 151
152 ivanceras/svgbob Convert your ascii diagram scribbles into happy little SVG rs 54.9% $17.37 139
153 crowdagger/crowbook Converts books written in Markdown to HTML, LaTeX/PDF and EPUB rs 54.3% $24.10 188
154 jesseduffield/lazygit simple terminal UI for git commands go 53.0% $10.12 108
155 go-critic/go-critic The most opinionated Go source code linter for code audit. go 52.7% $32.98 209
156 Stranger6667/jsonschema A high-performance JSON Schema validator for Rust rs 52.2% $17.30 124
157 sigoden/argc A Bash CLI framework, also a Bash command runner. rs 52.2% $48.62 226
158 rvben/rumdl Fast Markdown linter and formatter written in Rust rs 49.5% $21.14 165
159 rochacbruno/marmite Markdown makes sites - A Static Site Generator for Blogs rs 49.2% $11.67 99
160 arq5x/bedtools2 bedtools - the swiss army knife for genome arithmetic c 48.9% $17.50 144
161 madler/pigz A parallel implementation of gzip for modern multi-processor, multi-core machines. c 47.9% $21.11 140
162 paradigmxyz/solar Blazingly fast, modular and contributor friendly Solidity compiler, written in Rust rs 47.4% $25.00 193
163 bellard/quickjs Public repository of the QuickJS Javascript Engine. c 46.8% $18.15 110
164 tree-sitter/tree-sitter An incremental parsing system for programming tools rs 46.5% $17.26 145
165 stacked-git/stgit Stacked Git rs 46.4% $30.01 214
166 axodotdev/oranda ๐ŸŽ generate beautiful landing pages for your developer tools rs 43.5% $14.98 127
167 tukaani-project/xz XZ Utils c 41.4% $17.38 108
168 mgechev/revive ๐Ÿ”ฅ ~6x faster, stricter, configurable, extensible, and beautiful drop-in replacement for golint go 40.0% $46.19 235
169 mikefarah/yq yq is a portable command-line YAML, JSON, XML, CSV, TOML, HCL and properties processor go 38.9% $16.89 113
170 ducaale/xh Friendly and fast tool for sending HTTP requests rs 37.2% $45.04 227
171 dandavison/delta A syntax-highlighting pager for git, diff, grep, rg --json, and blame output rs 35.8% $14.66 131
172 OSGeo/PROJ PROJ - Cartographic Projections and Coordinate Transformations Library cpp 34.4% $13.09 105
173 boyter/scc Sloc, Cloc and Code: scc is a very fast accurate code counter with complexity calculations and COCOMO estimates written in pure Go go 31.0% $14.64 125
174 doxygen/doxygen Official doxygen git repository c 29.7% $10.90 115
175 johnkerl/miller Miller is like awk, sed, cut, join, and sort for name-indexed data such as CSV, TSV, and tabular JSON go 28.8% $22.19 192
176 chirlu/sox SoX, Swiss Army knife of sound processing c 24.9% $21.77 128
177 hpjansson/chafa ๐Ÿ“บ๐Ÿ—ฟ Terminal graphics for the 21st century. c 24.0% $11.77 96
178 Nukesor/pueue :stars: Manage your shell commands. rs 21.3% $17.68 119
179 danmar/cppcheck static analysis of C/C++ code cpp 19.3% $4.94 67
180 samtools/samtools Tools (written in C using htslib) for manipulating next-generation sequencing data c 18.0% $21.15 142
181 php/php-src The PHP Interpreter c 17.8% $27.39 167
182 ast-grep/ast-grep โšกA CLI tool for code structural search, lint and rewriting. Written in Rust rs 17.3% $13.83 116
183 typst/typst A markup-based typesetting system that is powerful and easy to learn. rs 16.8% $20.14 159
184 jgm/pandoc Universal markup converter hs 14.6% $59.55 320
185 OSGeo/gdal GDAL is an open source MIT licensed translator library for raster and vector geospatial data formats. cpp 12.8% $17.95 146
186 tstack/lnav Log file navigator cpp 11.3% $14.06 137
187 alecthomas/chroma A general purpose syntax highlighter in pure Go go 11.1% $9.48 84
188 universal-ctags/ctags A maintained ctags implementation c 10.6% $31.11 206
189 gromacs/gromacs Public/backup repository of the GROMACS molecular simulation toolkit. Please do not mine the metadata blindly; we use https://gitlab.com/gromacs/gromacs for code review and issue tracking. cpp 7.6% $8.71 87
190 ip7z/7zip 7-Zip cpp 5.9% $17.93 163
191 tinycc/tinycc Unofficial mirror of mob development branch c 4.3% $16.40 119
192 FFmpeg/FFmpeg Mirror of https://git.ffmpeg.org/ffmpeg.git c 3.7% $12.79 76
193 duckdb/duckdb DuckDB is an analytical in-process SQL database management system cpp 3.3% $28.66 170
194 jqlang/jq Command-line JSON processor c 1.0% $40.74 179
195 stathissideris/ditaa ditaa is a small command-line utility that can convert diagrams drawn using ascii art ('drawings' that contain characters that resemble lines like | / - ), into proper bitmap graphics. java 0.8% $17.60 134
196 sqlite/sqlite Official Git mirror of the SQLite source tree c 0.8% $21.76 135
197 parcel-bundler/lightningcss An extremely fast CSS parser, transformer, bundler, and minifier written in Rust. rs 0.7% $18.78 118
198 google/brotli Brotli compression format c 0.7% $28.02 173
199 KSXGitHub/parallel-disk-usage Highly parallelized, blazing fast directory tree analyzer rs 0.0% $24.77 162
200 sharkdp/hyperfine A command-line benchmarking tool rs 0.0% $16.99 115

Click row to see task details