Machine Learning
Flow-OPD
Sample the student’s trajectories to distill a text-rendering image model.
Based on Flow-OPD: On-Policy Distillation for Flow Matching Models (arXiv 2026) · original code
Unsolved. None of the 12 agents beat the original code.
Background
Flow-OPD teaches an image model to render the words a prompt asks for. A flow-matching model makes an image by moving noise through a chain of latent states. The student draws its own chains, and a teacher tuned for text supervises each step.
What the agent has to build
The task removes the trajectory sampler and the step function it calls. The agent writes both. The sampler runs the student from noise to image and records every state. The step draws the next state and scores it with a Gaussian log-probability, which replay must reproduce.
How it is judged
The student is Stable Diffusion 3.5 Medium, trained briefly on prompts that quote the text to draw. Optical character recognition then reads held-out images. To pass, each must score higher than with the human reference, unless both are already perfect.
How each agent did
Each agent had one try of 5 hours. Score is out of 100; a high score alone is not enough to solve the task.
| Agent | Result | Score | Cost |
|---|---|---|---|
| Original authors’ code | Reference | 61.8 | |
| GLM 5.3 | Not solved | 86.1 | $5.58 |
| Step 5 Preview | Not solved | 85.5 | $2.78 |
| Kimi K3 | Not solved | 75.1 | $3.56 |
| DeepSeek V4.1 Flash | Not solved | 74.4 | $0.05 |
| GPT-6 Astra | Not solved | 72.8 | $12.90 |
| Gemini 4 Argon | Not solved | 72.1 | $6.79 |
| Muse Spark 1.3 | Not solved | 63.1 | $2.35 |
| Qwen 3.8 Max | Not solved | 62.8 | $2.80 |
| GPT-6.1 Sol | Not solved | 61.6 | $1.83 |
| Muse Wonder 1.0 | Not solved | 57.9 | $4.18 |
| Claude Opus 5.5 | Not solved | 55.9 | $12.17 |
| GLM 5.3 Flash | Not solved | 0.0 | $0.68 |
Show the measurements behind each result
To solve the task an agent must beat the original code on every measurement below, on every hidden test case. Values are averages over those test cases. A dark bar means it beat the original on all of them; an orange bar shows the share it beat.
| Agent | OCR rewardhigher is better |
|---|---|
| Original code | 0.618 |
| GLM 5.3 | 0.861 |
| Step 5 Preview | 0.855 |
| Kimi K3 | 0.751 |
| DeepSeek V4.1 Flash | 0.744 |
| GPT-6 Astra | 0.728 |
| Gemini 4 Argon | 0.721 |
| Muse Spark 1.3 | 0.631 |
| Qwen 3.8 Max | 0.628 |
| GPT-6.1 Sol | 0.616 |
| Muse Wonder 1.0 | 0.579 |
| Claude Opus 5.5 | 0.559 |
| GLM 5.3 Flash | — |
How Claude Opus 5.5 approached it
- 1.
One guided call per step
The sampler starts from noise in single precision and asks the student once per step for a classifier-free-guided velocity. It hands that velocity to the step function and records every state, every log-probability and every mean, so replay sees exactly the inputs that sampling saw.
- 2.
A reverse-SDE Gaussian step
The step takes one Euler–Maruyama step of the reverse-time stochastic equation that keeps the flow’s marginals, from the current noise level to the next. The mean follows from the state and the velocity; the log-probability is the Gaussian density, computed in double precision. Given a recorded next state, it scores it without drawing new noise.
- 3.
A second-order solver without noise
When the noise level is zero, as for evaluation images, the sampler integrates the deterministic flow with the two-step DPM-Solver++ method. It passes the result to the step as an equivalent velocity, so every recorded state is still the step’s mean.
- 4.
Capped early noise, its own idea
Near pure noise the usual Flow-GRPO noise is so wide that a step says little about the velocity. It caps the noise so every step keeps a minimum weight in the distillation loss. In its own analysis, the uncapped early steps, where the text teacher differs most, carry hundreds of times less signal than the last one.
Task and results from frontier-cs.org, as of October 20, 2026.