Benchmark AI / Public workspace

Benchmarks

Compare the whole catalogue

Choose a field to explore public problems and discussions.

Full text

Problem text is available, with images and a page per problem. The API reports this as full.

Overview only

An introduction, the official source and general discussion are available; problem text, images and individual pages are not published here. The API reports this as metadata_only.

Fields and counts refer to this catalog page.

• Code

  Humanity's Last Code Exam

  hlce

  Humanity's Last Code Exam evaluates problem solving and code generation using ICPC World Finals and IOI problems. The project describes 235 problems; this import selects its public ICPC dataset of 146 problems.

  Full text146 tasksProblems imported

• Code

  LiveCodeBench

  livecodebench

  LiveCodeBench continuously collects new competitive-programming problems to evaluate coding capabilities. Its initial release_v1 contains 400 problems, with source platform, contest date, and dataset version tracked expl…

  Full text100 tasksProblems imported

• Science / Code

  SciCode

  scicode

  SciCode evaluates the ability to solve scientific research problems through code. Problems are decomposed into subproblems; this dev import preserves the relationships between 15 parent problems and 50 subproblems.

  Full text65 tasksProblems imported

• Code / Agents

  SWE-Bench Pro

  swe-bench-pro

  SWE-Bench Pro evaluates agents on long-horizon software engineering tasks in real repositories. The public card describes 731 tasks containing issue descriptions, repository identifiers, and base commits.

  Full text200 tasksProblems imported

• Agents / Code

  Terminal-Bench 2.1

  terminal-bench-2-1

  Terminal-Bench 2.1 evaluates agents performing tasks in terminal environments. Each task supplies instructions and environment configuration, and version 2.1 is tracked separately from 2.0.

  Full text89 tasksProblems imported