Robotics
Diffusion Policy
Sample a robot’s actions from a diffusion policy.
Based on Diffusion Policy: Visuomotor Policy Learning via Action Diffusion (RSS 2023) · original code
Unsolved. None of the 12 agents beat the original code.
Background
Diffusion Policy controls a robot by generating its moves the way image models generate pictures. It starts from random noise and removes it step by step. A trained network guides each step, using what the robot just saw. The robot makes the first few moves, then plans again.
What the agent has to build
The task removes the sampler, the loop that turns noise into moves. The trained network stays fixed. The agent writes the loop: how many denoising steps, and how each step updates the sample. The authors’ loop takes 100 steps.
How it is judged
A simulated robot pushes a T-shaped block into place over many episodes. To pass, it must match or beat the human reference on reward and success in every group of episodes. It must also plan faster and finish the whole run sooner.
How each agent did
Each agent had one try of 5 hours. Score is out of 100; a high score alone is not enough to solve the task.
| Agent | Result | Score | Cost |
|---|---|---|---|
| Original authors’ code | Reference | 13.4 | |
| GLM 5.3 | Not solved | 59.8 | $4.07 |
| Claude Opus 5.5 | Not solved | 57.1 | $11.14 |
| Muse Wonder 1.0 | Not solved | 55.4 | $8.04 |
| Step 5 Preview | Not solved | 54.1 | $5.38 |
| DeepSeek V4.1 Flash | Not solved | 53.3 | $0.04 |
| Gemini 4 Argon | Not solved | 52.8 | $10.41 |
| GPT-6.1 Sol | Not solved | 52.5 | $1.32 |
| GPT-6 Astra | Not solved | 52.0 | $15.01 |
| Kimi K3 | Not solved | 50.9 | $1.45 |
| Qwen 3.8 Max | Not solved | 47.9 | $2.71 |
| Muse Spark 1.3 | Not solved | 44.5 | $1.80 |
| GLM 5.3 Flash | Not solved | 15.9 | $0.54 |
Show the measurements behind each result
To solve the task an agent must beat the original code on every measurement below, on every hidden test case. Values are averages over those test cases. A dark bar means it beat the original on all of them; an orange bar shows the share it beat.
| Agent | Wall time, whole workload (s)lower is better | Success ratehigher is better | Mean best reward per episodehigher is better | Action prediction time (ms)lower is better |
|---|---|---|---|---|
| Original code | 54.7 | 0.25 | 0.924 | 1,220 |
| GLM 5.3 | 10.8 | 0.25 | 0.94 | 42.5 |
| Claude Opus 5.5 | 11.8 | 0.167 | 0.921 | 60.4 |
| Muse Wonder 1.0 | 12.9 | 0.333 | 0.935 | 87.2 |
| Step 5 Preview | 14.2 | 0.417 | 0.966 | 132 |
| DeepSeek V4.1 Flash | 14.5 | 0.25 | 0.956 | 103 |
| Gemini 4 Argon | 14.9 | 0.333 | 0.953 | 123 |
| GPT-6.1 Sol | 13.6 | 0.25 | 0.896 | 94.8 |
| GPT-6 Astra | 13.6 | 0.333 | 0.87 | 45 |
| Kimi K3 | 15.9 | 0.25 | 0.951 | 86.1 |
| Qwen 3.8 Max | 18.6 | 0.333 | 0.979 | 204 |
| Muse Spark 1.3 | 19 | 0.167 | 0.951 | 247 |
| GLM 5.3 Flash | 42.9 | 0 | 0.763 | 809 |
How Claude Opus 5.5 approached it
- 1.
Five network calls
Instead of the human reference’s 100 denoising steps, the loop calls the fixed network only five times, at noise levels 95, 70, 45, 22 and 5 of its training schedule. Sampling starts from Gaussian noise at the first level.
- 2.
A clean guess at each level
At each level, the network’s noise estimate gives a guess of the clean move sequence, clipped to the range the scheduler allows. Before each call, the parts of the sequence fixed by the observations are written back in.
- 3.
Second-order jumps
Between levels, it follows the deterministic probability-flow path, measured in log signal-to-noise ratio, with the multistep DPM-Solver++ update. Each jump combines the current and the previous clean guesses, which makes it second order.
- 4.
A free correction
On reaching a new level, it corrects the point it landed on with the UniPC corrector, which uses the clean guess just made there, so the correction costs no extra network call. The last clean guess becomes the plan.
Task and results from frontier-cs.org, as of October 20, 2026.