Benchmark AI / Public workspace

Benchmarks

Compare the whole catalogue

Choose a field to explore public problems and discussions.

Full text

Problem text is available, with images and a page per problem. The API reports this as full.

Overview only

An introduction, the official source and general discussion are available; problem text, images and individual pages are not published here. The API reports this as metadata_only.

Fields and counts refer to this catalog page.

• Reasoning

  ARC-AGI-2

  arc-agi-2

  ARC-AGI-2 evaluates the ability to infer unfamiliar rules from a few examples and apply them to new grids. Training and evaluation are separate; this import covers the 120 public evaluation tasks.

  Full text120 tasksProblems imported

• Reasoning / Agents

  ARC-AGI-3

  arc-agi-3

  ARC-AGI-3 is an interactive benchmark for exploring unfamiliar game environments, learning their rules, and planning actions. The official toolkit provides an environment interface, with game versions and trial records t…

  Overview onlyNot imported on this site

• Reasoning

  BIG-Bench Extra Hard (BBEH)

  bbeh

  BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains 200 Boolean-expression questions from the full distribution.

  Full text200 tasksProblems imported

• Reasoning / Multimodal

  Humanity's Last Exam

  hle

  Humanity's Last Exam evaluates AI knowledge and reasoning across academic disciplines. It includes text and image questions; this catalog provides an introduction and official links in accordance with the publisher's req…

  Overview onlyNot imported on this site

• Reasoning / Long context

  LongBench v2

  longbench-v2

  LongBench v2 evaluates deep understanding and reasoning over long contexts through multiple-choice questions. Its official description lists 503 questions spanning tasks such as single-document and multi-document QA and …

  Full text200 tasksProblems imported

• Reasoning

  ZebraLogicBench

  zebralogicbench

  ZebraLogicBench evaluates reasoning on logic puzzles that require satisfying clues to determine attribute assignments. The official card lists 1,000 grid-mode puzzles and a separate multiple-choice setting.

  Full text200 tasksProblems imported