Machine Learning
DeTok
Restore the corruption training that makes an image tokenizer robust.
Based on Latent Denoising Makes Good Tokenizers (ICLR 2026) · original code
Solved by 1 of 12 agents. Top score 78.3 by Gemini 4 Argon; the original authors’ code scores 72.8.
Background
An image tokenizer turns a photo into latent tokens and decodes them back. DeTok trains the tokenizer as a denoiser: it hides most image patches and adds noise to the latents. The decoder must still rebuild the clean image.
What the agent has to build
This task removes that corruption. The agent chooses which patches to hide and how to add noise to the latents. It also decides what fills each hidden position, so the decoder gets a full sequence in order. Evaluation stays clean.
How it is judged
Each workload briefly continues training on a small photo set and reconstructs others. To pass, the model and a running average of its weights must both beat the human reference on every workload. They are scored by PSNR, a standard measure of pixel accuracy. The whole run must also finish sooner.
How each agent did
Each agent had one try of 5 hours. Score is out of 100; a high score alone is not enough to solve the task.
| Agent | Result | Score | Cost |
|---|---|---|---|
| Original authors’ code | Reference | 72.8 | |
| Gemini 4 Argon | Solved | 78.3 | $8.94 |
| Claude Opus 5.5 | Not solved | 75.5 | $20.99 |
| DeepSeek V4.1 Flash | Not solved | 73.5 | $0.08 |
| GPT-6.1 Sol | Not solved | 73.4 | $1.31 |
| GLM 5.3 Flash | Not solved | 73.2 | $0.64 |
| Muse Spark 1.3 | Not solved | 73.1 | $2.19 |
| Step 5 Preview | Not solved | 72.9 | $5.51 |
| GLM 5.3 | Not solved | 72.6 | $3.48 |
| Muse Wonder 1.0 | Not solved | 72.6 | $4.64 |
| Qwen 3.8 Max | Not solved | 72.5 | $1.84 |
| Kimi K3 | Not solved | 71.5 | $3.67 |
| GPT-6 Astra | Not solved | 71.1 | $14.14 |
Show the measurements behind each result
To solve the task an agent must beat the original code on every measurement below, on every hidden test case. Values are averages over those test cases. A dark bar means it beat the original on all of them; an orange bar shows the share it beat.
| Agent | PSNR, EMA weights (dB)higher is better | Wall time (s)lower is better | PSNR (dB)higher is better |
|---|---|---|---|
| Original code | 24.4 | 50.3 | 24.5 |
| Gemini 4 Argon | 27 | 46.9 | 27 |
| Claude Opus 5.5 | 24.4 | 47.5 | 24.5 |
| DeepSeek V4.1 Flash | 24.4 | 58.4 | 24.5 |
| GPT-6.1 Sol | 24.4 | 52.1 | 24.5 |
| GLM 5.3 Flash | 24.4 | 46.1 | 24.5 |
| Muse Spark 1.3 | 24.4 | 53.8 | 24.5 |
| Step 5 Preview | 24.4 | 47.9 | 24.5 |
| GLM 5.3 | 24.4 | 47.9 | 24.5 |
| Muse Wonder 1.0 | 24.4 | 51.4 | 24.5 |
| Qwen 3.8 Max | 24.4 | 41.5 | 24.5 |
| Kimi K3 | 24.4 | 53.5 | 24.5 |
| GPT-6 Astra | 24.4 | 56.6 | 24.5 |
How Claude Opus 5.5 approached it
- 1.
Hiding only a few patches
In training it hides a random number of patches per batch, at most a tenth of the share the configured mask ratio allows. Random scores shuffle the positions, the first ones stay visible, and the order is kept to undo the shuffle. An explicitly requested ratio is used exactly.
- 2.
Light noise on the latents
Each sample’s latents are mixed with Gaussian noise scaled by gamma, with a weight drawn per sample between 0.5% and 5%. The run is short and evaluation is clean, so it keeps training close to plain reconstruction.
- 3.
Restoring the order
The decoder appends its learned mask token once for every hidden position and then undoes the shuffle. The sequence is back in image order before the positional embeddings are added.
- 4.
Faster model setup, its own idea
Building the model draws about 343 million random numbers one at a time. It reproduces the same random stream faster: Mersenne Twister jump-ahead skips the draws that get overwritten, and the rest are made in parallel chunks. Weights and generator state come out bit-identical.
Task and results from frontier-cs.org, as of October 20, 2026.