Benchmark AI / Public workspace

Benchmarks

Compare the whole catalogue

Choose a field to explore public problems and discussions.

Full text

Problem text is available, with images and a page per problem. The API reports this as full.

Overview only

An introduction, the official source and general discussion are available; problem text, images and individual pages are not published here. The API reports this as metadata_only.

Fields and counts refer to this catalog page.

• Reasoning / Agents

  ARC-AGI-3

  arc-agi-3

  ARC-AGI-3 is an interactive benchmark for exploring unfamiliar game environments, learning their rules, and planning actions. The official toolkit provides an environment interface, with game versions and trial records t…

  Overview onlyNot imported on this site

• Agents / Search

  BrowseComp

  browsecomp

  BrowseComp evaluates web browsing and the ability to identify answers from multiple clues. This catalog supports introductions and discussion of search methods without republishing decrypted questions or answers.

  Overview onlyNot imported on this site

• Agents / Search

  BrowseComp-Plus

  browsecomp-plus

  BrowseComp-Plus supports evaluation of retrieval methods using BrowseComp tasks and a search corpus. This catalog links to the official distribution without republishing plaintext questions or corpus content.

  Overview onlyNot imported on this site

• Agents / Multimodal

  GAIA

  gaia

  GAIA evaluates general AI assistants on tasks involving information gathering and tool use. Some tasks include attachments; this catalog provides an introduction in view of access and resharing restrictions.

  Overview onlyNot imported on this site

• Code / Agents

  SWE-Bench Pro

  swe-bench-pro

  SWE-Bench Pro evaluates agents on long-horizon software engineering tasks in real repositories. The public card describes 731 tasks containing issue descriptions, repository identifiers, and base commits.

  Full text200 tasksProblems imported

• Agents / Code

  Terminal-Bench 2.1

  terminal-bench-2-1

  Terminal-Bench 2.1 evaluates agents performing tasks in terminal environments. Each task supplies instructions and environment configuration, and version 2.1 is tracked separately from 2.0.

  Full text89 tasksProblems imported