Architecture
KAMI
Write the double-precision block matrix-multiply kernel for one GPU.
Based on KAMI: Communication-Avoiding General Matrix Multiplication within a Single GPU (SC 2025) · original code
Solved by 6 of 12 agents. Top score 23 by Claude Opus 5.5; the original authors’ code scores 17.8.
Background
KAMI multiplies small matrices inside one GPU thread block, a group of threads on one processor. It brings communication-avoiding methods from distributed matrix multiplication into the block. Its warps split the work so they exchange little data, and compute on the tensor cores.
What the agent has to build
This task removes KAMI’s double-precision kernel, the code each thread block runs. The agent writes it. Each block computes A times B transposed, plus half of the old C, on its own matrices. Every output element is checked.
How it is judged
Each workload launches thousands of blocks, each with its own matrices, in shapes from 16×16×24 to 128×64×64. To pass, the kernel must beat the human reference on throughput for every shape, by more than the measured timing noise.
How each agent did
Each agent had one try of 5 hours. Score is out of 100; a high score alone is not enough to solve the task.
| Agent | Result | Score | Cost |
|---|---|---|---|
| Original authors’ code | Reference | 17.8 | |
| Claude Opus 5.5 | Solved | 23.0 | $10.52 |
| Gemini 4 Argon | Solved | 23.0 | $7.81 |
| GPT-6 Astra | Solved | 22.5 | $4.28 |
| Kimi K3 | Solved | 22.4 | $1.60 |
| GPT-6.1 Sol | Solved | 21.7 | $1.24 |
| GLM 5.3 | Solved | 21.5 | $9.62 |
| Muse Wonder 1.0 | Not solved | 21.7 | $8.27 |
| Qwen 3.8 Max | Not solved | 21.0 | $6.58 |
| DeepSeek V4.1 Flash | Not solved | 19.9 | $0.14 |
| GLM 5.3 Flash | Not solved | 19.1 | $0.53 |
| Muse Spark 1.3 | Not solved | 16.6 | $5.38 |
| Step 5 Preview | Not solved | 0.0 | $3.60 |
Show the measurements behind each result
To solve the task an agent must beat the original code on every measurement below, on every hidden test case. Values are averages over those test cases. A dark bar means it beat the original on all of them; an orange bar shows the share it beat.
| Agent | TFLOPShigher is better |
|---|---|
| Original code | 4.84 |
| Claude Opus 5.5 | 6.54 |
| Gemini 4 Argon | 6.56 |
| GPT-6 Astra | 6.34 |
| Kimi K3 | 6.27 |
| GPT-6.1 Sol | 6.11 |
| GLM 5.3 | 5.96 |
| Muse Wonder 1.0 | 6.01 |
| Qwen 3.8 Max | 5.64 |
| DeepSeek V4.1 Flash | 5.16 |
| GLM 5.3 Flash | 5.38 |
| Muse Spark 1.3 | 4.06 |
| Step 5 Preview | — |
How Claude Opus 5.5 solved it
- 1.
Streaming the inputs
With one small, independent multiplication per block, the batch is bound by memory bandwidth, so the kernel is built to keep memory busy. It copies A and B into shared memory with asynchronous 16-byte copies: all of K at once when they fit in 48 KiB, otherwise in a multi-stage pipeline over K.
- 2.
Fusing the old C
Each thread loads its part of C straight into its accumulator registers and scales it by one half before the products start, so the read of C overlaps the traffic of A and B. Results leave the accumulators as 16-byte streaming stores.
- 3.
Double-precision tensor cores
The products use the FP64 tensor-core instruction on 16×8×8 fragments. At compile time, it lays the warps over the output fragments so that as many warps as divide the grid evenly take part, with the fewest shared-memory loads per product.
- 4.
Conflict-free shared loads
Inside each 8-wide step of K, it remaps the two K positions a thread needs onto adjacent ones, in both A and B. The sum is unchanged, but each pair becomes one 16-byte load, and a swizzle of the rows keeps simultaneous reads on different memory banks.
- 5.
Reverse launch order, its own idea
Blocks work through the batch from the end: block b takes the b-th slice from the last. The first wave of blocks thus starts on the most recently written end of the batch, which is still in cache. Every slice is still computed exactly once.
Task and results from frontier-cs.org, as of October 20, 2026.