• Stick to the train budget. Training must finish within either 12 or 192 B200-hours, timed on a single HGX B200 node with 8×B200 SXM GPUs (180 GB HBM3e). The timer starts after the first training step, and no training step may be taken once the budget is exhausted.
• Do not train on SWE-bench. Training data may not be derived from SWE-bench's 12 source repositories¹ or their forks and mirrors: their source code, git history, issues, pull requests, or agent trajectories generated from them.

All else is fair game: architecture, optimizer, data, scaffolding, inference-time strategy, and so on.

¹ astropy/astropy, django/django, matplotlib/matplotlib, mwaskom/seaborn, pallets/flask, psf/requests, pydata/xarray, pylint-dev/pylint, pytest-dev/pytest, scikit-learn/scikit-learn, sphinx-doc/sphinx, and sympy/sympy.

Submissions are made via pull request, containing your changes to the training stack along with:

• The run's log file, which lets us verify that the run was produced with the PR's training-stack commit, its training time, and the hash of the model weights.
• A link to the training data (if changed): training data must be publicly available.
• A link to the final model weights.
• The SWE-bench Verified eval trajectory files (5 independent samples per problem).

See here for an example. The main branch corresponds to the 192 B200-hour speedrun; a separate branch hosts the 12 B200-hour one. Once we verify that a submission improves on the previous best, the pull request is merged and the records are updated.

Contributions are welcome even if you can't train runs at scale:

• Training speed improvements (e.g., kernels) that do not affect numerics, and thus buy more training within the same budget.
• Promising $60-scale submissions will be scaled up to $1,000.
• If the training data is unchanged and your run improves held-out perplexity on teacher trajectories, we can run the pass@1 evals.

We only care about maximizing SWE-bench Verified performance, so long as that performance is not due to test-set leakage. Hence the strict rules on not training on the SWE-bench repositories. Fortunately, it is straightforward to create SWE-bench-like test sets, which lets us periodically check for adaptivity to the test set.

The downside of B200-hours is, of course, that it fixes the hardware. However, GPU-hours is a less gameable constraint than training FLOPs: a very sparse MoE, for example, may use relatively few FLOPs yet take much longer to train on a single node.