A community speedrun of SWE-bench Verified:
the best agent that $1000 can buy, trained from scratch.
- Claude 3 Opus 15.8%
- GPT-4o 23.2%
- Claude 3.5 Sonnet 33.6%
$1000 track (192 B200-hours)
| Date | Description | pass@1 | Model | Evals | Train |
|---|---|---|---|---|---|
| 2026-10-06 | + smaller model, web warmup | 25.5% WR | weights ↗ | results ↗ | script ↗ |
| 2026-10-02 | + nemotron lightning trajs | 15.7% | weights ↗ | results ↗ | script ↗ |
| 2026-10-02 | + fixed vLLM serving | 11.8% | weights ↗ | results ↗ | script ↗ |
| 2026-08-12 | nanochat + long context + qwen coder trajs | 11.0% | weights ↗ | results ↗ | script ↗ |
$60 track (12 B200-hours)
Think you can improve on it? Read the rules and submit a run.