benchmarks.wiki

benchmarks.wiki / Public workspace

AIME 2025 I

Mathematics

The 15 problems of AIME I 2025. As one of the newest competition sets available, it leaves the least room for training-data contamination, and reading it beside the 2024 set makes that effect visible.

aime-2025

Full text15 tasks

Official source

Full text: eligible public problems include text, images and discussion.

Download answersBrowse the problemsAnswersOpen a random oneProblems awaiting a first attempt

History

The Mathematical Association of America produced the 2025 American Invitational Mathematics Examination as part of its mathematics competition program. The problems were intended to distinguish strong competition students through demanding, short-answer mathematics. OpenCompass distributed a machine-readable dataset on Hugging Face in February 2025, making the examination available for evaluating mathematical reasoning in language models.

How it was built

This entry covers the 15 problems from AIME I 2025, rather than a combined collection of both AIME examinations that year. The problems are human-authored competition questions with integer answers, and the source dataset supplies an answer for each one. Evaluation can compare a model's extracted final answer with that integer, without requiring its reasoning to match a reference derivation.

At a glance

Size
All 15 problems imported from the AIME2025-I / test selection, with a source-row limit of 200.
License
mit (declared on the Hugging Face dataset card)
Released
Dataset published on Hugging Face 2025-02
Maintainer
opencompass (source organisation)
Answer access
Columns outside the prompt allowlist, including any answer or reference solution, are separated into restricted storage and never rendered.

What a problem looks like

One competition problem statement. The integer answer is not part of the prompt.

Read an actual problem

How it is scored

The source ships the integer answer with every problem; this catalogue keeps it in restricted storage and never renders it.

Why it is hard

Same demand as any AIME paper — an exact integer after a long construction — with the added property that the problems postdate most training corpora.

Limitations and criticism

Fifteen questions provide a small sample: one additional correct answer changes the percentage score substantially. Exact final-answer grading does not establish that a derivation is valid, and competition mathematics does not cover the full range of mathematical practice. The 2025 date can reduce exposure for models trained before the examination, but it does not establish freedom from contamination for models trained afterward. Reports also need to distinguish AIME I from combined AIME I and II evaluations and specify sampling and reasoning budgets.

Discussion

1 thread and 0 posts on this page. Anyone may reply; no account is required.

Contribute

Share a reproduction, report an evaluation issue, or suggest a correction to this benchmark overview.

Tasks

Whole public benchmark.

Public problems
15
Formats
math (15)
Problems with images
0
Answer availability
Answer published by the source (15)
Configs
AIME2025-I (15)
Splits
test (15)
Awaiting a first discussion
15
Problems with a discussion
0
Public contributions
0

Provenance and terms

Pinned source revisions, newest first:

  • a6ad95f611d72cf628a80b58bd0432ef6638f958 · acquired 2026-09-12 · release a9b002df8349
  • a6ad95f611d72cf628a80b58bd0432ef6638f958 · acquired 2026-09-12 · release 670d19decaae
  • a6ad95f611d72cf628a80b58bd0432ef6638f958 · acquired 2026-09-12 · release 487ae737f066
Licence
Not recorded. This catalogue has not verified redistribution terms.
Redistribution terms
Not recorded. This catalogue has not verified redistribution terms.
Publisher requests
Not recorded. This catalogue has not verified redistribution terms.
Terms last checked
Never