benchmarks.wiki / Public workspace
Download Benchmark Answers
Answer sets for every benchmark in this catalogue, one download each.
Each row is the answer set held for one benchmark. Availability describes what the upstream source distributes; no answer value is shown on this page, and the problem pages stay readable without one.
Download all answer sets
サインアップしてくださいSign up to download answer sets.
This catalogue has not finished recording the licence and redistribution terms of these sources, so downloads stay closed while that is outstanding. Browse the benchmarks · Compare the catalogue
| Benchmark | Field | Problems | Source answers | Answer set |
|---|---|---|---|---|
| AgentHarm | Safety · Agents | — | Overview only | DownloadサインアップしてくださいSign up to download answer sets. |
| AIME 2024 | Mathematics | 30 | Published by the source (30) | DownloadサインアップしてくださいSign up to download answer sets. |
| AIME 2025 I | Mathematics | 15 | Published by the source (15) | DownloadサインアップしてくださいSign up to download answer sets. |
| AMC (AIMO validation set) | Mathematics | 83 | Published by the source (83) | DownloadサインアップしてくださいSign up to download answer sets. |
| ARC-AGI-2 | Reasoning | 120 | Published by the source (120) | DownloadサインアップしてくださいSign up to download answer sets. |
| ARC-AGI-3 | Reasoning · Agents | — | Overview only | DownloadサインアップしてくださいSign up to download answer sets. |
| ARC-Challenge | Reasoning · Science | 200 | Published by the source (200) | DownloadサインアップしてくださいSign up to download answer sets. |
| Arena-Hard-Auto | Instruction following | — | Overview only | DownloadサインアップしてくださいSign up to download answer sets. |
| AssistantBench | Agents · Search | 181 | Withheld by the source (181) | DownloadサインアップしてくださいSign up to download answer sets. |
| Berkeley Function Calling Leaderboard | Agents · Tools | — | Overview only | DownloadサインアップしてくださいSign up to download answer sets. |
| BIG-Bench Extra Hard (BBEH) | Reasoning | 200 | Published by the source (200) | DownloadサインアップしてくださいSign up to download answer sets. |
| BIG-Bench Hard | Reasoning | — | Overview only | DownloadサインアップしてくださいSign up to download answer sets. |
| BigCodeBench | Code | 200 | Published by the source (200) | DownloadサインアップしてくださいSign up to download answer sets. |
| BoolQ | Reasoning | 200 | Published by the source (200) | DownloadサインアップしてくださいSign up to download answer sets. |
| BrowseComp | Agents · Search | — | Overview only | DownloadサインアップしてくださいSign up to download answer sets. |
| BrowseComp-Plus | Agents · Search | — | Overview only | DownloadサインアップしてくださいSign up to download answer sets. |
| CodeContests | Code | — | Overview only | DownloadサインアップしてくださいSign up to download answer sets. |
| CritPt | Science | 70 | Published by the source (70) | DownloadサインアップしてくださいSign up to download answer sets. |
| CRUXEval | Code · Reasoning | 200 | Published by the source (200) | DownloadサインアップしてくださいSign up to download answer sets. |
| DROP | Reasoning | 200 | Published by the source (200) | DownloadサインアップしてくださいSign up to download answer sets. |
| DS-1000 | Code · Science | 200 | Published by the source (200) | DownloadサインアップしてくださいSign up to download answer sets. |
| FrontierMath | Mathematics | — | Overview only | DownloadサインアップしてくださいSign up to download answer sets. |
| FrontierScience | Science | 100 | Published by the source (100) | DownloadサインアップしてくださいSign up to download answer sets. |
| GAIA | Agents · Multimodal | — | Overview only | DownloadサインアップしてくださいSign up to download answer sets. |
| GPQA | Science | — | Overview only | DownloadサインアップしてくださいSign up to download answer sets. |
| GSM8K | Mathematics | 200 | Published by the source (200) | DownloadサインアップしてくださいSign up to download answer sets. |
| HarmBench | Safety | — | Overview only | DownloadサインアップしてくださいSign up to download answer sets. |
| HellaSwag | Reasoning | 200 | Published by the source (200) | DownloadサインアップしてくださいSign up to download answer sets. |
| HumanEval | Code | 164 | Published by the source (164) | DownloadサインアップしてくださいSign up to download answer sets. |
| HumanEval+ | Code | 164 | Published by the source (164) | DownloadサインアップしてくださいSign up to download answer sets. |
| Humanity's Last Code Exam | Code | 146 | Published by the source (146) | DownloadサインアップしてくださいSign up to download answer sets. |
| Humanity's Last Exam | Reasoning · Multimodal | — | Overview only | DownloadサインアップしてくださいSign up to download answer sets. |
| IFEval | Reasoning · Instruction following | 200 | Not recorded (200) | DownloadサインアップしてくださいSign up to download answer sets. |
| JailbreakBench | Safety | — | Overview only | DownloadサインアップしてくださいSign up to download answer sets. |
| LiveCodeBench | Code | 100 | External grader only (100) | DownloadサインアップしてくださいSign up to download answer sets. |
| LongBench v2 | Reasoning · Long context | 200 | Published by the source (200) | DownloadサインアップしてくださいSign up to download answer sets. |
| MATH | Mathematics | 200 | Published by the source (200) | DownloadサインアップしてくださいSign up to download answer sets. |
| MathVista | Mathematics · Multimodal | — | Overview only | DownloadサインアップしてくださいSign up to download answer sets. |
| MBPP | Code | 200 | Published by the source (200) | DownloadサインアップしてくださいSign up to download answer sets. |
| MBPP+ | Code | 200 | Published by the source (200) | DownloadサインアップしてくださいSign up to download answer sets. |
| MGSM | Mathematics · Multilingual | 200 | Published by the source (200) | DownloadサインアップしてくださいSign up to download answer sets. |
| MMLU | Reasoning · Knowledge | 200 | Published by the source (200) | DownloadサインアップしてくださいSign up to download answer sets. |
| MMLU-Pro | Reasoning | 200 | Published by the source (200) | DownloadサインアップしてくださいSign up to download answer sets. |
| MMMU | Multimodal | — | Overview only | DownloadサインアップしてくださいSign up to download answer sets. |
| MMMU-Pro | Multimodal | 200 | Published by the source (200) | DownloadサインアップしてくださいSign up to download answer sets. |
| MT-Bench | Instruction following | — | Overview only | DownloadサインアップしてくださいSign up to download answer sets. |
| MuSR | Reasoning | 200 | Published by the source (200) | DownloadサインアップしてくださいSign up to download answer sets. |
| OlympiadBench | Mathematics · Science · Multimodal | 200 | Published by the source (200) | DownloadサインアップしてくださいSign up to download answer sets. |
| Omni-MATH | Mathematics | 200 | Published by the source (200) | DownloadサインアップしてくださいSign up to download answer sets. |
| OSWorld | Agents · Multimodal | — | Overview only | DownloadサインアップしてくださいSign up to download answer sets. |
| RULER | Reasoning · Long context | — | Overview only | DownloadサインアップしてくださいSign up to download answer sets. |
| SciCode | Science · Code | 65 | Published by the source (65) | DownloadサインアップしてくださいSign up to download answer sets. |
| SimpleQA | Knowledge · Search | 200 | Published by the source (200) | DownloadサインアップしてくださいSign up to download answer sets. |
| SWE-bench Lite | Code · Agents | — | Overview only | DownloadサインアップしてくださいSign up to download answer sets. |
| SWE-Bench Pro | Code · Agents | 200 | External grader only (200) | DownloadサインアップしてくださいSign up to download answer sets. |
| SWE-bench Verified | Code · Agents | — | Overview only | DownloadサインアップしてくださいSign up to download answer sets. |
| tau-bench | Agents | — | Overview only | DownloadサインアップしてくださいSign up to download answer sets. |
| Terminal-Bench 2.1 | Agents · Code | 89 | External grader only (89) | DownloadサインアップしてくださいSign up to download answer sets. |
| TruthfulQA | Reasoning · Safety | 200 | Published by the source (200) | DownloadサインアップしてくださいSign up to download answer sets. |
| WebArena | Agents · Search | — | Overview only | DownloadサインアップしてくださいSign up to download answer sets. |
| WinoGrande | Reasoning | — | Overview only | DownloadサインアップしてくださいSign up to download answer sets. |
| ZebraLogicBench | Reasoning | 200 | Withheld by the source (200) | DownloadサインアップしてくださいSign up to download answer sets. |