What it measures
- Complex engineering tasks
- Agentic coding
- Repository-scale issue resolution
SWE-bench Pro
A professional software-engineering agent benchmark with public and private leaderboard splits.
How to read the score
% resolved; higher is better.
Sourced Ranking
| Rank | Model | Provider | Score | Source | Source Type | Verified |
|---|---|---|---|---|---|---|
| 1 | OpenAI GPT-5.4 | OpenAI | 57.7 | Introducing GPT-5.4 Accessed 2026-06-11 | official | Yes |
| 2 | OpenAI GPT-5.2 Thinking | OpenAI | 55.6 | Introducing GPT-5.2 Accessed 2026-06-11 | official | Yes |
| 3 | Gemini 3.1 Pro | 54.2 | Gemini 3.1 Pro model page Accessed 2026-06-11 | official | Yes |
Limitations
- Do not merge public and private split scores.
- The paper is static; Scale's leaderboard is dynamic.
- Separate public and private splits in result records.
leaderboard
Scale Labs SWE-bench Pro leaderboard
The paper is static; Scale's leaderboard is dynamic. Separate public and private splits in result records.
paper
SWE-bench Pro paper or specification
Use for benchmark definition, not necessarily for latest model scores.
third-party
SWE-bench Pro dataset or code
Use for dataset, code, or implementation details; score freshness depends on the benchmark.