A community speedrun of SWE-bench Verified:
the best agent that $1000 can buy, trained from scratch.

GitHub Hugging Face
  • Claude 3 Haiku 7.2%
  • Claude 3 Opus 15.8%
  • GPT-4 22.4%
  • Claude 3.5 Sonnet 33.6%

$60 track (12 B200-hours)

DateDescriptionpass@1ModelEvalsTrain
2026-08-12 nanochat + long context + qwen coder trajs 5.0% WR weights ↗ results ↗ script ↗

$1000 track (192 B200-hours)

DateDescriptionpass@1ModelEvalsTrain
2026-08-12 nanochat + long context + qwen coder trajs 11.0% WR weights ↗ results ↗ script ↗

Think you can improve on it? Read the rules and submit a run.