Database
Spectrum
Make EVM stack checkpoints cheap to create and restore.
Based on Spectrum: Speedy and Strictly-Deterministic Smart Contract Transactions for Blockchain Ledgers (VLDB 2024) · original code
Solved by 1 of 12 agents. Top score 88.5 by Claude Opus 5.5; the original authors’ code scores 61.3.
Background
Spectrum runs Ethereum smart-contract transactions in parallel, yet commits them in the order consensus fixed. When two transactions conflict, it does not restart the later one from scratch. It rolls that transaction back to a checkpoint taken partway through and runs on from there.
What the agent has to build
A checkpoint saves where the program is and every value on its operand stack. This task removes the stack that makes these checkpoints cheap. The agent writes it: save a checkpoint, restore one, and keep many alive at once without any of them changing another.
How it is judged
Each submission must first restore thousands of checkpoints exactly, checked against a simple reference. It then runs three standard transaction workloads: YCSB, SmallBank and a TPC-C-style order mix. To pass, it must beat the human reference on four counts: throughput, checkpoint creation time, checkpoint restore time and memory per checkpoint.
How each agent did
Each agent had one try of 5 hours. Score is out of 100; a high score alone is not enough to solve the task.
| Agent | Result | Score | Cost |
|---|---|---|---|
| Original authors’ code | Reference | 61.3 | |
| Claude Opus 5.5 | Solved | 88.5 | $26.20 |
| Muse Wonder 1.0 | Not solved | 80.6 | $15.88 |
| DeepSeek V4.1 Flash | Not solved | 80.3 | $0.13 |
| GLM 5.3 | Not solved | 78.6 | $12.20 |
| GPT-6.1 Sol | Not solved | 75.8 | $8.62 |
| GPT-6 Astra | Not solved | 75.0 | $19.01 |
| Qwen 3.8 Max | Not solved | 24.8 | $5.03 |
| Kimi K3 | Not solved | 9.1 | $6.79 |
| Step 5 Preview | Not solved | 4.1 | $10.00 |
| GLM 5.3 Flash | Not solved | 3.5 | $2.98 |
| Muse Spark 1.3 | Not solved | 2.3 | $13.41 |
| Gemini 4 Argon | Not solved | 2.3 | $24.55 |
Show the measurements behind each result
To solve the task an agent must beat the original code on every measurement below, on every hidden test case. Values are averages over those test cases. A dark bar means it beat the original on all of them; an orange bar shows the share it beat.
| Agent | Checkpoint creation time (ns)lower is better | Checkpoint restore time (ns)lower is better | Committed transactions per secondhigher is better | Correctnesshigher is better | Memory per checkpointlower is better |
|---|---|---|---|---|---|
| Original code | 231 | 65 | 42,500 | 1 | 13,400 |
| Claude Opus 5.5 | 43 | 25 | 48,100 | 1 | 8,560 |
| Muse Wonder 1.0 | 60 | 38 | 38,000 | 1 | 8,230 |
| DeepSeek V4.1 Flash | 39 | 319 | 25,100 | 1 | 4,460 |
| GLM 5.3 | 39 | 428 | 47,900 | 1 | 4,300 |
| GPT-6.1 Sol | 119 | 31 | 44,000 | 1 | 8,960 |
| GPT-6 Astra | 82 | 35 | 46,400 | 1 | 11,800 |
| Qwen 3.8 Max | 1,230 | 55 | 28,400 | 1 | 43,400 |
| Kimi K3 | 1,800 | 755 | 40,700 | 1 | 35,100 |
| Step 5 Preview | 1,550 | 739 | 35,900 | 1 | 37,200 |
| GLM 5.3 Flash | 4,410 | 756 | 34,200 | 1 | 42,600 |
| Muse Spark 1.3 | 2,190 | 717 | 37,400 | 1 | 43,300 |
| Gemini 4 Argon | 2,150 | 719 | 37,700 | 1 | 43,300 |
How Claude Opus 5.5 solved it
- 1.
Pages of four items
The stack stores its items in pages of four, and a page table lists the pages in order. A checkpoint holds the code position and a reference to a page table, so saving a deep stack copies no items.
- 2.
Copy on write
Saving a deep stack shares the live page table in constant time. The live stack copies the table, and then each page, only when it first writes there, so every checkpoint keeps the values it saw. A new checkpoint is registered lazily, on the next change, which keeps saving short.
- 3.
Flat copies for shallow stacks
A stack of at most 63 items is simply copied into a self-contained block. That checkpoint then depends on nothing, and the live stack keeps writing its own pages in place.
- 4.
Constant-time restore
Restoring one of the stack’s own checkpoints swaps its page table back in and raises a reference count, again in constant time. A flat copy is written back item by item, and a checkpoint from another stack is copied in.
- 5.
Memory inside the stack space
Pages and tables come from free lists carved out of the execution state’s own preallocated stack space, and fall back to the heap only when that runs out. A mark-and-sweep pass frees pages that no table refers to, and reads take a direct path while the bottom pages lie contiguously in memory.
Task and results from frontier-cs.org, as of October 20, 2026.