benchmarks.wiki / Public workspace

What does downloading CodeContests answers involve?

Every benchmark in this catalogue, and where to get its answers.

Benchmarks63
Public problems91,222
Answers published by their source89,108
Scored by a harness389
Solutions written here241

Most of these benchmarks publish their own answer keys, linked from each benchmark's page, and where a benchmark is scored by running a submission through its harness, the harness is the destination. This site does not redistribute those keys.

What the Download button here offers is one file of answers per benchmark. Taking it needs an account — an ID and a password, nothing else — and this site asks you to contribute a model in exchange. The button opens a panel on this page that says exactly what is asked and what you get.

Every benchmark in the catalogue, ordered by name
BenchmarkFieldProblemsAnswersDownload
AgentHarmSafety · Agents—Full answers (solved by benchmarks.wiki)Download72 MB
AIME 2024Mathematics30Full answers (solved by benchmarks.wiki)Download53 MB
AIME 2025 IMathematics15Full answers (solved by benchmarks.wiki)Download50 MB
AMC (AIMO validation set)Mathematics83Full answers (solved by benchmarks.wiki)Download34 MB
ARC-AGI-2Reasoning120Full answers (solved by benchmarks.wiki)Download76 MB
ARC-AGI-3Reasoning · Agents—Full answers (solved by benchmarks.wiki)Download71 MB
ARC-ChallengeReasoning · Science1,172Full answers (solved by benchmarks.wiki)Download48 MB
ARC-EasyReasoning · Science2,376Full answers (solved by benchmarks.wiki)Download69 MB
Arena-Hard-AutoInstruction following—Full answers (solved by benchmarks.wiki)Download63 MB
AssistantBenchAgents · Search181Full answers (solved by benchmarks.wiki)Download57 MB
Berkeley Function Calling LeaderboardAgents · Tools—Full answers (solved by benchmarks.wiki)Download63 MB
BIG-Bench Extra Hard (BBEH)Reasoning4,513Full answers (solved by benchmarks.wiki)Download67 MB
BIG-Bench HardReasoning6,507Full answers (solved by benchmarks.wiki)Download64 MB
BigCodeBenchCode1,139Full answers (solved by benchmarks.wiki)Download38 MB
BoolQReasoning3,270Full answers (solved by benchmarks.wiki)Download45 MB
BrowseCompAgents · Search—Full answers (solved by benchmarks.wiki)Download67 MB
BrowseComp-PlusAgents · Search—Full answers (solved by benchmarks.wiki)Download55 MB
CodeContestsCode—Full answers (solved by benchmarks.wiki)Download59 MB
CritPtScience70Full answers (solved by benchmarks.wiki)Download36 MB
CRUXEvalCode · Reasoning800Full answers (solved by benchmarks.wiki)Download44 MB
DROPReasoning9,535Full answers (solved by benchmarks.wiki)Download81 MB
DS-1000Code · Science1,000Full answers (solved by benchmarks.wiki)Download84 MB
FrontierMathMathematics—Full answers (solved by benchmarks.wiki)Download32 MB
FrontierScienceScience100Full answers (solved by benchmarks.wiki)Download51 MB
GAIAAgents · Multimodal—Full answers (solved by benchmarks.wiki)Download26 MB
GPQAScience—Full answers (solved by benchmarks.wiki)Download47 MB
GSM8KMathematics1,319Full answers (solved by benchmarks.wiki)Download50 MB
HarmBenchSafety—Full answers (solved by benchmarks.wiki)Download33 MB
HellaSwagReasoning10,042Full answers (solved by benchmarks.wiki)Download57 MB
HumanEvalCode164Full answers (solved by benchmarks.wiki)Download36 MB
HumanEval+Code164Full answers (solved by benchmarks.wiki)Download34 MB
Humanity's Last Code ExamCode146Full answers (solved by benchmarks.wiki)Download74 MB
Humanity's Last ExamReasoning · Multimodal—Full answers (solved by benchmarks.wiki)Download66 MB
IFEvalReasoning · Instruction following541Full answers (solved by benchmarks.wiki)Download47 MB
JailbreakBenchSafety—Full answers (solved by benchmarks.wiki)Download69 MB
LiveCodeBenchCode100Full answers (solved by benchmarks.wiki)Download63 MB
LongBench v2Reasoning · Long context200Full answers (solved by benchmarks.wiki)Download76 MB
MATHMathematics5,000Full answers (solved by benchmarks.wiki)Download91 MB
MathVistaMathematics · Multimodal—Full answers (solved by benchmarks.wiki)Download39 MB
MBPPCode1,401Full answers (solved by benchmarks.wiki)Download53 MB
MBPP+Code378Full answers (solved by benchmarks.wiki)Download70 MB
MGSMMathematics · Multilingual2,750Full answers (solved by benchmarks.wiki)Download66 MB
MMLUReasoning · Knowledge14,015Full answers (solved by benchmarks.wiki)Download56 MB
MMLU-ProReasoning12,032Full answers (solved by benchmarks.wiki)Download91 MB
MMMUMultimodal—Full answers (solved by benchmarks.wiki)Download29 MB
MMMU-ProMultimodal200Full answers (solved by benchmarks.wiki)Download68 MB
MT-BenchInstruction following—Full answers (solved by benchmarks.wiki)Download71 MB
MuSRReasoning756Full answers (solved by benchmarks.wiki)Download76 MB
OlympiadBenchMathematics · Science · Multimodal200Full answers (solved by benchmarks.wiki)Download30 MB
Omni-MATHMathematics4,406Full answers (solved by benchmarks.wiki)Download65 MB
OSWorldAgents · Multimodal—Full answers (solved by benchmarks.wiki)Download24 MB
RULERReasoning · Long context—Full answers (solved by benchmarks.wiki)Download53 MB
SciCodeScience · Code65Full answers (solved by benchmarks.wiki)Download57 MB
SimpleQAKnowledge · Search4,326Full answers (solved by benchmarks.wiki)Download68 MB
SWE-bench LiteCode · Agents—Full answers (solved by benchmarks.wiki)Download65 MB
SWE-Bench ProCode · Agents200Full answers (solved by benchmarks.wiki)Download66 MB
SWE-bench VerifiedCode · Agents—Full answers (solved by benchmarks.wiki)Download65 MB
tau-benchAgents—Full answers (solved by benchmarks.wiki)Download27 MB
Terminal-Bench 2.1Agents · Code89Full answers (solved by benchmarks.wiki)Download49 MB
TruthfulQAReasoning · Safety817Full answers (solved by benchmarks.wiki)Download45 MB
WebArenaAgents · Search—Full answers (solved by benchmarks.wiki)Download27 MB
WinoGrandeReasoning—Full answers (solved by benchmarks.wiki)Download39 MB
ZebraLogicBenchReasoning1,000Full answers (solved by benchmarks.wiki)Download66 MB