Database
FARGO Query Core
Write the multi-probing query core of an inner-product search.
Based on FARGO: Fast Maximum Inner Product Search via Global Multi-Probing (VLDB 2023) · original code
Solved by 8 of 12 agents. Top score 52.6 by Claude Opus 5.5; the original authors’ code scores 50.9.
Background
FARGO finds the vectors with the largest inner product with a query, without checking them all. It hashes each vector into several tables, one bit per random projection. It probes the closest buckets across all tables at once. It stops when better answers become unlikely.
What the agent has to build
This task removes the query core, and the agent writes it. The core streams buckets from all tables, each listed once. Each carries its distance from the query: the sum of squared projections on the bits that differ. The core also decides when to stop.
How it is judged
Searches run on two datasets from the paper, Audio and MNIST, at twelve settings. To pass, every setting must equal or beat the human reference on recall and answer quality. It must also verify no more candidates and answer faster.
How each agent did
Each agent had one try of 5 hours. Score is out of 100; a high score alone is not enough to solve the task.
| Agent | Result | Score | Cost |
|---|---|---|---|
| Original authors’ code | Reference | 50.9 | |
| Claude Opus 5.5 | Solved | 52.6 | $10.05 |
| GPT-6 Astra | Solved | 52.5 | $3.96 |
| Gemini 4 Argon | Solved | 52.5 | $6.16 |
| GLM 5.3 | Solved | 52.2 | $2.37 |
| Muse Wonder 1.0 | Solved | 52.2 | $10.98 |
| Muse Spark 1.3 | Solved | 52.0 | $7.31 |
| Qwen 3.8 Max | Solved | 52.0 | $1.66 |
| GPT-6.1 Sol | Solved | 51.9 | $0.67 |
| Kimi K3 | Not solved | 52.1 | $2.90 |
| DeepSeek V4.1 Flash | Not solved | 52.0 | $0.12 |
| GLM 5.3 Flash | Not solved | 52.0 | $0.88 |
| Step 5 Preview | Not solved | 0.0 | $9.21 |
Show the measurements behind each result
To solve the task an agent must beat the original code on every measurement below, on every hidden test case. Values are averages over those test cases. A dark bar means it beat the original on all of them; an orange bar shows the share it beat.
| Agent | Candidates verifiedlower is better | Overall ratiohigher is better | Query time (ms)lower is better | Recallhigher is better |
|---|---|---|---|---|
| Original code | 7,060 | 0.983 | 9.48 | 0.76 |
| Claude Opus 5.5 | 7,060 | 0.983 | 6.64 | 0.76 |
| GPT-6 Astra | 7,060 | 0.983 | 6.56 | 0.76 |
| Gemini 4 Argon | 7,060 | 0.983 | 6.26 | 0.76 |
| GLM 5.3 | 7,060 | 0.983 | 7.32 | 0.76 |
| Muse Wonder 1.0 | 7,060 | 0.983 | 6.3 | 0.76 |
| Muse Spark 1.3 | 7,060 | 0.983 | 7.27 | 0.76 |
| Qwen 3.8 Max | 7,060 | 0.983 | 6.69 | 0.76 |
| GPT-6.1 Sol | 7,060 | 0.983 | 7.03 | 0.76 |
| Kimi K3 | 7,060 | 0.983 | 7.5 | 0.76 |
| DeepSeek V4.1 Flash | 7,060 | 0.983 | 7.07 | 0.76 |
| GLM 5.3 Flash | 7,060 | 0.983 | 7.26 | 0.76 |
| Step 5 Preview | — | — | — | — |
How Claude Opus 5.5 solved it
- 1.
Distances from flip costs
In each table, flipping a bit costs the squared projection of the query on that bit. A bucket’s distance is the sum of the costs of the bits where it differs from the query’s own key, so the query’s own bucket in every table comes first, at 0.
- 2.
A heap for the first probes
Early buckets come from a small heap over all tables. It grows sets of flipped bits by shift and expand moves over each table’s sorted costs, so every bucket appears once, in ascending distance, with ties going to the lower table.
- 3.
Sorted batches for deep probes
Deeper probing switches to batches: each table’s sorted costs are split into a low and a high half with precomputed sorted subset sums, and every bucket below the next threshold is emitted at once, then sorted. Both phases compute the same sum, so the handoff is exact.
- 4.
The stop rule
From the k-th best inner product so far, the approximation ratio and the bound on the largest inner product, it reads the paper’s calibration table. It stops when the k-th result reaches the bound, or when the chance that more probes across all tables help falls below 0.1. For fixed inputs this is a cutoff distance, cached until they change.
Task and results from frontier-cs.org, as of October 20, 2026.