Programming Languages
Glenside
Write the analysis that tracks tensor shapes through Glenside’s rewrites.
Based on Pure Tensor Program Rewriting via Access Patterns (Representation Pearl) (MAPS 2021) · original code
Solved by 9 of 12 agents. Top score 91 by GPT-6 Astra; the original authors’ code scores 82.8.
Background
Glenside is a language for tensor programs that maps them onto matrix hardware. Each tensor is an access pattern: an outer shape of items, plus each item’s shape. Rewrites fill an e-graph with equivalent programs, and a fixed rule spots forms a systolic-array accelerator can run.
What the agent has to build
This task removes the analysis that attaches facts to each class of equal expressions: shapes, known zero regions and accelerator use. The agent writes how a fact is built from its parts’ facts, and how two facts merge when classes join.
How it is judged
Workloads are small convolutions and concatenations feeding a dot product. To pass, on every workload it must find an exact accelerator form whenever the human reference does. It must also use no more rewrite time and add no more graph nodes.
How each agent did
Each agent had one try of 5 hours. Score is out of 100; a high score alone is not enough to solve the task.
| Agent | Result | Score | Cost |
|---|---|---|---|
| Original authors’ code | Reference | 82.8 | |
| GPT-6 Astra | Solved | 91.0 | $6.98 |
| GPT-6.1 Sol | Solved | 91.0 | $0.92 |
| Claude Opus 5.5 | Solved | 91.0 | $14.67 |
| Gemini 4 Argon | Solved | 91.0 | $6.51 |
| Kimi K3 | Solved | 90.8 | $8.20 |
| DeepSeek V4.1 Flash | Solved | 90.6 | $0.08 |
| GLM 5.3 | Solved | 90.5 | $16.72 |
| Muse Wonder 1.0 | Solved | 90.5 | $5.98 |
| GLM 5.3 Flash | Solved | 90.1 | $1.58 |
| Muse Spark 1.3 | Not solved | 82.9 | $4.23 |
| Qwen 3.8 Max | Not solved | 0.0 | $9.35 |
| Step 5 Preview | Not solved | 0.0 | $19.19 |
Show the measurements behind each result
To solve the task an agent must beat the original code on every measurement below, on every hidden test case. Values are averages over those test cases. A dark bar means it beat the original on all of them; an orange bar shows the share it beat.
| Agent | E-graph growthlower is better | Finds im2colhigher is better | Rewrite time (µs)lower is better |
|---|---|---|---|
| Original code | 55,200 | 1 | 181,000 |
| GPT-6 Astra | 17.4 | 1 | 113 |
| GPT-6.1 Sol | 18.1 | 1 | 116 |
| Claude Opus 5.5 | 17.4 | 1 | 110 |
| Gemini 4 Argon | 18.1 | 1 | 110 |
| Kimi K3 | 18.6 | 1 | 175 |
| DeepSeek V4.1 Flash | 21.9 | 1 | 244 |
| GLM 5.3 | 26.4 | 1 | 279 |
| Muse Wonder 1.0 | 26.4 | 1 | 301 |
| GLM 5.3 Flash | 41 | 1 | 468 |
| Muse Spark 1.3 | 55,200 | 1 | 167,000 |
| Qwen 3.8 Max | — | — | — |
| Step 5 Preview | — | — | — |
How Claude Opus 5.5 solved it
- 1.
A fact for every operator
For each kind of node, the analysis builds its fact from its children’s facts: the outer and item shapes of an access pattern, the output size of windows and pooling, and broadcast shapes. A Cartesian product, for example, joins the outer shapes and pairs the items.
- 2.
Marking accelerator use
Systolic-array nodes, their blocked and convolution forms, and explicit accelerator calls get the shape of the product they compute and are marked as containing accelerator use. Every parent inherits the mark, so a class knows whether an accelerator form lies inside.
- 3.
Merging facts
When two classes join, the merge checks that both facts cover the same number of elements. A settled shape wins over an unsettled one, accelerator use is kept if either side has it, and the merge reports exactly which side changed, so the e-graph revisits only what moved.
- 4.
Rewrites to matrix form, its own idea
It also changes two rewrites. A dot product of a Cartesian product is rebuilt over flattened operands, so the accelerator rule sees a plain matrix product. A concatenation of two such dot products becomes one dot product of block operands, zero-padded so the extra products add nothing. Each rule skips classes it has already rewritten.
Task and results from frontier-cs.org, as of October 20, 2026.