SWE-bench Verified
A curated SWE-bench split for resolving real software issues with agentic coding systems.
Coding Ranking
Signals for code generation, debugging, ML engineering, and terminal-based software agent work.
Main rankings require at least 3 of 5 included sourced benchmark rows.
Within the main ranking, rows are sorted by average normalized score; coverage is shown separately and used only as a tie-breaker.
Missing benchmark data is not treated as zero; models below the threshold appear under limited evidence.
| Rank | Model | Provider | Score | Coverage | Source | Source Type | Verified | Last Updated |
|---|---|---|---|---|---|---|---|---|
| 1 | Gemini 3.1 Pro | 67.8 | 3/5 | Gemini 3.1 Pro model page Accessed 2026-06-11 | official | Yes | 2026-06-11 |
Limited Evidence
| Model | Provider | Score | Coverage | Freshest Source | Last Updated |
|---|---|---|---|---|---|
| Claude Opus 4.6 | Anthropic | 81.4 | 1/5 | Introducing Claude Opus 4.6 Accessed 2026-06-11 | 2026-06-11 |
| Gemini 3 Pro | 76.2 | 1/5 | Gemini 3.1 Pro model page Accessed 2026-06-11 | 2026-06-11 | |
| OpenAI GPT-5.2 Thinking | OpenAI | 67.8 | 2/5 | Introducing GPT-5.2 Accessed 2026-06-11 | 2026-06-11 |
| OpenAI GPT-5.4 | OpenAI | 57.7 | 1/5 | Introducing GPT-5.4 Accessed 2026-06-11 | 2026-06-11 |
| DeepSeek-R1 | DeepSeek | 49.2 | 1/5 | DeepSeek-R1 model card Accessed 2026-06-11 | 2026-06-11 |
SWE-bench Pro
A professional software-engineering agent benchmark with public and private leaderboard splits.
Terminal-Bench 2.x
A terminal-environment agent benchmark for command-line task completion.
MLE-bench
A machine-learning engineering benchmark based on Kaggle-style competition tasks.
TerminalWorld-Verified
A terminal-agent benchmark candidate that needs source confirmation.