GPQA Diamond
95.0 pass@1
Full 198 question set. Held out split 97.1, 95% CI 94.2-99.3.
Assay
Measured, not claimed.
Every figure on this page comes from our own runs, on public instruments, with a frozen harness and a confidence interval on every cell. Where a measurement does not exist, the page prints the gap instead of filling it.
Three instruments, three headline scores.
AIME
99.0 pass@1
30 runs per problem. majority@30 = 100, CI 97.0-100.
LiveCodeBench
89.6 pass@1
91.8 at pass@5, scored by the official evaluator.
The measurement behind the page
- 39,000+ scored measurements
- 13 engines
- 95% bootstrap CIs on every cell
- 41 safety refusals scored as failures
- contamination counter-tested
Range
Where the frontier sits, and where our figure lands.
Each axis runs from 80 to 100. The grey band is the span of the published third-party figures on that instrument. The teal mark is our own measurement.
GPQA Diamond
pass@1
- Span of published third-party figures 92.5-95.45
- Yasi One 95.0
- Held out split 97.1
AIME
pass@1, no tools
- Published third-party figures, no tools 92-100
- Yasi One 99.0
- 95% CI 97.0-100
LiveCodeBench
pass@1, matching conditions
- Yasi One 89.6
- No comparable public disclosure under matching conditions.
Scoreboard
Our line is a reference, not a rank.
Under each instrument the public list is sorted strictly by score, from the highest lower bound down, and every line carries the class of source it came from. Our own line sits apart, as a reference: we did not re-measure anyone else's figure, so we claim no place among them.
Figures as of August 2026.
GPQA Diamond pass@1
| Model | Score | Source |
|---|---|---|
| Yasi One NOT RANKED | 95.0 full 198 question set 97.1 held out split (95% CI 94.2-99.3) | our run |
| Gemini 3.1 Pro Preview | 94.1-95.45 | indep. |
| GPT-5.6 Sol | 94.1-95.2 | indep. |
| Kimi K3 | 93.5 | aggreg. |
| GPT-5.5 | 93.5 | aggreg. |
| Claude Opus 5 | 93.2-93.4 | aggreg. |
| Grok 4.5 | 92.9-93.1 | aggreg. |
| Gemini 3.6 Flash | 92.8-93.4 | aggreg. |
| Claude Fable 5* | 92.6-93.2 | aggreg. |
| GPT-5.6 Terra | 92.5-92.9 | aggreg. |
One public evaluator also publishes a refusal corrected variant, where some scores differ materially. Every Yasi figure counts refusals as failures, from day one.
97.1 on the held out split sits above every published value, and 95.0 on the full set sits in the top of the independent range, whose upper edge is 95.45.
AIME pass@1, no tools, saturated instrument
| Model | Score | Source |
|---|---|---|
| Yasi One NOT RANKED | 99.0 majority@30 = 100 | our run |
| GLM-5.2 | 99.2 | vendor |
| GPT-5.2 Pro / Codex | 98.7-100 | aggreg. |
| Claude Opus / Fable / Mythos variants | 95-99.8 | system cards |
| Gemini 3 / 3.1 variants | 95-100 | vendor |
| Grok 4 / 4.5 variants | ~92-100 | aggreg. |
Parity at the ceiling. The instrument is saturated, so what limits the figures here is the test, not the engines.
LiveCodeBench pass@1
Independent snapshots, August 12, 2026, sorted by score.
| Model | Score | Source |
|---|---|---|
| Gemini 3 Pro Preview | 91.7 | indep. |
| Gemini 3 Flash Preview | 90.8 | indep. |
| Claude Fable 5 | 89.8 | aggreg. |
| Yasi One NOT RANKED | 89.6 our run, official evaluator, declared conditions | our run |
| DeepSeek V3.2 Speciale | 89.6 | indep. |
| Kimi K2.6 | 89.6 | aggreg. |
| Claude Opus 5 | 89.0 | aggreg. |
| Grok 4.5 | 87.3 | aggreg. |
| GPT-5.x Codex | 87-88 | aggreg. |
Vendor reported claims, not verified by a third party.
| Model | Score | Source |
|---|---|---|
| DeepSeek V4 Pro | 93.5 | vendor |
| Qwen 3.7 Max | 91.6 | vendor |
Top tier. In the independent block two values sit above our run, and one aggregator places Claude Fable 5 at 89.8, which is 0.2 above. Two unverified vendor claims sit higher still. Conditions are declared only on our line.
Sources named: Artificial Analysis, Vals.ai, BenchLM and pricepertoken, on their August 2026 snapshots, together with vendor disclosures and system cards. These figures were not re-measured by us. Product and model names belong to their owners; no endorsement is implied.
How to read these lines.
These are our own runs, and third party replay is invited. Read the lines for what they are: a routing platform's line is a maximum over thirteen engines, where a laboratory's line is one model.
Platform depth
Thirteen engines, one routed answer.
Each dot is one engine on the frozen set. The dots carry no labels, and the front line, the figure the router serves, is marked in teal.
GPQA Diamond
full set
Front line 95.0
AIME
no tools
Front line 99.0
LiveCodeBench
pass@1
Front line 89.6
Per engine detail is available to partners under agreement.
Refusal ledger
Forty one refusals, every one counted as a failure.
Engines sometimes decline a sensitive science question on safety grounds. No answer arrives, so nothing can be marked correct, and the honest score is a zero. Forty one such refusals were recorded in this campaign, and every one of them was scored as a failure against our own result rather than quietly removed from the denominator. The router covers the gap the only way that is defensible: it puts the strongest engine willing to take the task on that task.
Contamination
Every score above 98 was challenged.
A score above 98 triggers the memorisation rule: the result is presumed to be recall until a counter test says otherwise. The counter test cuts the frozen set in two, problems published in 2025 and problems published in 2026, after the training windows in question. Deltas across the partition stayed small, and one flagged engine scored higher on the 2026 problems, which is the signature of capability rather than recall. The flags stay on record either way, because a test that only ever clears its subject is not a test.
Speed of answer
Depth without the usual wait.
Science
10.9 s
median (p50) to a complete answer
Mathematics
18.6 s
median (p50), while holding 99.0 across thirty repetitions
Depth and speed are usually a trade. The job of the router is to refuse that trade, task by task.
Routing
The task goes to the router. The answer goes wherever it measures best.
-
The router takes the task
Every request enters through the Yasi router, which decides where the work belongs before any engine sees it.
-
Exact answers go to one engine
On instruments with a single correct answer, the task goes to the strongest single engine.
-
The Majlis council takes the open questions
The council serves open ended reasoning and writing, where no single key decides the outcome.
Stated plainly: on exact answer instruments, single engine routing holds the frontier, and the council multiplies cost without adding exact accuracy.
The council, measured against a single engine.
Published, not hidden. The verdict rule was fixed before the runs.
| Run | Single | Council | Paired delta | 95% CI | Cost | Verdict |
|---|---|---|---|---|---|---|
| GPQA test, 138 items | 97.1 | 92.8 | -4.4 | [-8.0, -1.5] | x5.6 | TIE |
| AIME dev, 30 x 30 | 100.0 | 97.3 | -2.7 | [-4.2, -1.4] | x4.0 | TIE |
| AIME frozen test, council: NOT MEASURED. Printed as a gap, never estimated. | ||||||
- TIE by pre-fixed rule.
Method
Six rules, fixed before the first run.
-
Public instruments
Every instrument is public and maintained by a third party. No private set, and no in-house benchmark marked by the author of the thing it measures.
-
Frozen harness
One harness for every engine and every run, frozen at a named upstream commit with a fixed seed, so a run can be reproduced instead of recounted.
-
Declared conditions
Conditions are declared line by line. Where no comparison exists under matching conditions the page prints the absence, and a measurement we did not run is printed as a gap, never estimated, interpolated or inferred.
-
Statistics, not vibes
Every headline figure carries a 95% bootstrap confidence interval, and the rule that turns a measured difference into a verdict was fixed before the runs rather than chosen once the numbers were in. That is how a difference we could measure is still published as a tie.
-
Dataset integrity
Scores above the memorisation threshold are challenged on a partition by year of publication, and safety refusals are counted as failures instead of leaving the denominator.
-
Replayable
Harness, upstream commit, seed and cost are recorded together, so every run behind this page can be replayed and costed to the cent.
Next campaign
Phase 2 of the measurements.
The instrument set widens. The next campaign takes on the work that a single question and a single answer cannot hold, and it runs under the rules already kept above: a frozen harness, conditions declared, refusals counted as failures, absences printed instead of filled. No date is announced on this page. The figures will be published when the runs are done, whichever way they fall.
- Long-horizon software engineering
- Terminal and computer use
- Long-context retrieval
- A factuality instrument
Glossary
The words on this page, defined.
- pass@1
- One attempt per problem, marked right or wrong. No retry, no best of several.
- majority@30
- Thirty independent attempts per problem. The answer that comes back most often is the one marked.
- pass@5
- Five attempts per problem. The problem counts as solved if any one attempt is correct.
- 95% CI
- A 95% bootstrap confidence interval: the range the figure would land in nineteen times out of twenty if the same run were repeated.
- p50
- The median. Half the answers arrive faster than this, half slower.
- NOT MEASURED
- We did not run it. The gap is printed as a gap, never estimated, interpolated or inferred.
Measured on Yasi One infrastructure July 28 to August 10, 2026, on public instruments with a frozen in-house harness derived from lm-evaluation-harness 0.4.12 (upstream commit 6d64254, seed 20260728). The comparison figures in the scoreboard come from public evaluators, vendor disclosures and system cards as of August 2026, unadjusted and not re-measured by us, with no third-party endorsement. Benchmark names belong to their respective maintainers. No benchmark item is reproduced on this page.