benchmarks.wiki / Public workspace
.·::--=+********++===--------::··.. ..·:=+**#####*+==-::······:::------------------::::······::--==++**++=-:··.
...···::---------:::··········... ..·::--=====---::························::::::···:::::::--==+**##*+=-::·.
................... ....·········.......... ...........::--::::--==+**##%#*+=-:·.
··--==------==+*##%%#*+=-:·.
··=-+==-----==+*#%%%#*+=-:·.
:·==*+=------=+*#%%@%#+=-:·.
::+=*+=--::--=+*##%%%#+=-:·.
::+=*+=-::::--=+*#%%%*+=-:·.
::+=*+=-:::::-=+*##%#*+=-·..
..··:::::::::··. ..··::::::··.........·······==*=-::···::-=+*#%#*+=:·..
... ..... .··:-+*********=-·. .·-=********+=-:··..···:--==+===+=-:·····:-=+*###*+-:·.
..... ...·······...··::-+*#@@@@@@@@@#+-·. ·-+#@@@@@@@@@%*=-:··.··:-=+*####*+=-:··..··:-=+*#*+=-:·.
.·:-=++*##**++=======++++++==-:·... ...·:=+#%@@@@@@%*+=:·.. ..·::-==++++++++++===--:::·················:·····...
.··::--===---:::::::::---:::··... ...·:--=+++++==-::·... ....···:::::::::::::::·····..·······:::::::::::····..
....······................. ....········..... .......................·.······:::-------:::···..
..·······:::---=====-:::··..
..··::··:::--========-::··..
..:·:::::::--==+++++=--:··..
·.::-:::::--==+++*+++=-::·..
··::-:::::--=++*****+=-::·..
··-:-:::::--=++*****+=--:·..
..···::::::::··. ..··:::::::···........·········-:--::::--=++*####*+=-:·..
...... ...·::-=+*******+=-·. ·:-+********+=-::····::--==++==--:-:::::--=++*####*+=-:·..
.......·····:::-=+*#%@@@@@@@#*=-·. .·=*%@@@@@@@@@@#+=-::::--=+*#%%##*+=-::::::-==+*####*+=-:·.
.··:------:····::--=++*++==-:·.....·:-=*#@@@@@@@@#+=:. .·:-=+*#%%%%%##**+==--:::······..... ..·····..
..··::::····.····::------::··......·::-=+++*++==-:·.. ..·::--========---::···.............. ...···...
..................·····..... .....········... .....········........ ......... ...····..
.................····..
................·····...
.....·····....········..
......··················..
.....····::::::::::::···..
....····::::::-----:::··..
....... ....···::::::··.. ..··::::::···................... ..·····::------===---::·..
......·...............··::--==++**++=--:·. .·-=+*******+=-::······:::----::···..····::---====+++==-::·..
.......··..·······:::--==++**##%%###*+=-:. ·:=*%@@@@@@@@@%*+--::::--=++++++=--::····:::-==++++***+=-::·..
...... ..·:-=++++=--:·....··:=+#@@@@@@@@#*=:. ·:=+#%@@@@@@@@%#*++=---::::::::··.. .·:----:·..
..... ..··::------::···.···::-==+++++=-::·. ..·:-=++******++=--::···.........·... .·::---:·..
....·······............······... ...·····::::·····..... ..... ..·::-::·.
... .·:::::·.
.. .·::::··.
.··:::··.
..··:::··..
.··::::··..
... ...··::::::··.
.......... ......·········... ..····::····.... ..........···:-----:··.
..·········.........···:::---=====--::·.. .·:-=++***++=--:··.............. .....·····:::--===--::·.
..····:::::····::::---====+++++++++==-::·. .·-=*%@@@@@@@%*+=-:········:::···... ....···:::---==+++==-::·.
.·:==++++==-:··...·:-=+#%@@@@@@#*=:. .:-+#@@@@@@@@@@@#*++==---====+==--:·. .:-+*###*=-:..
..·::------::·······::--======-:··. .·:-=+*********++=-::········::::····. .·-=*##*+=-:.
...········................... ...···::::::::::···.... .....···.. .·:=+***+=-·.
..·. .·:-+***+=:·.
.... .·-=+++=-:·.
.... .·:-=+==-:·.
. . .·:-====-:·.
. .·:-===--:·..
.·:-====-:··.
............ .................. ...·········..... ..·::-=+==--:·.
..···::::····.......·····::::------:::··... ..··:--==+===--:···...... ...··:-==++++=-:·.
.··:::::::::·::::::-------==========--::·... ...·:-=+*#%%%%#*+=-:··....... ......···::--=+****+=-:·.
.·:-=+*****+=-:··....·:-=+#%%@@%#*+-:. .·:=*%@@@@@@@@@@%*+==---==++****+=-:·. .·:=*#@@@%#*=:·.
..·::--===--::····.····::------::·.. .··:-+**********+=--::····:::-----::·.. .·-=*%@@@%#*=-·.
...·········.................. ...··:::::::::::···............·....·.. .·-=*%@@@%#*=-·.
..·.. .·-=*%@@@%#*=-·.
..·.. .·:=+#%@@%#*=:·.
..... .·:-+*%@@%#+=:·.
..... ..:-=*#%%#*+-:·.
... .·:=+*###*=-:·.
... ..·:-=+*#*+=-:·.
......... ..........................············.......... ... ..·:-=+*#*++-:·..
..·······............····::::------::::········:::--======---::::·····... ............··:-=+**##*+=-:·.
.··:::···········:::::----===++++++===-:::···::--=++**##**++==--::::···.. ..........···:--=+*####*+-:·..
..·:=+*#####*+=-:·.. ..·:-=+*#%%%#*+=:·. .·:=*#@@@@@@@@%*+-::···::-=+++++=-·. .:-+*%@@@@%*+-:·..
..·::--====--::··......···::----:::··.. ..·:-=+********+=-:··...···::----:··. ·:=*%@@@@@%#+=:·..
...········..... ......... ...···:::::::::···... .....··.... .·-+#@@@@@@%#*=:·.
.:=*%@@@@@@%#*=:·.
.·:=*%@@@@@@%#+-·.
. . ..·-=*%@@@@@@%#=:·.
. ... ..·:=*#@@@@@@%*=:.
. .......·:-+*#%@@@@#+-:.
.........·:-=+*#%%%#*=-:·.
.. ....................··························... ..........·...··::-=+*####*=-:·..
........ ....····:::-----------::::::----==================--::··. ...·······..······:--=+***#*+=-:·..
....... ....··:::---===++********++========++**#####**********++==-:·. .··:::::::::··:::--=++*****+=-:·..
..·:-=*#%%%##+=-:·. .·:-=*#%%%%%#*=-:.. .:-=*#%@@@%#*=-·.. .··:----:·. ·:=+*#%@@%%*+=-:·..
..··:-------::··... ...··::-------::·.. .·:-==+**++=-:··. ..··::··.. .:-+#@@@@@@@%*+-:·.
....·····...... ............ ..···:::::··... ....... ·-+#@@@@@@@@%#+-·.
.:=*@@@@@@@@@%*=:.
.:-+%@@@@@@@@@%+-·
..·:=*%@@@@@@@@@*=·.
...·:-+*%@@@@@@@@#+:.
. ..·::-=*#%@@@@@%#+-·.
..··:::-=++*##%##*+=-·..
...................·····················::::::···... ....····....··::::--=+++**++==-:·..
...···::::-----------------==========---====+++++++=-:··. ..··::-----:::··:::::---===++===-:··..
..·::--===+++***********+++++****######******##%%%@@%##+=-:·. ..·:-=++++++==---::::----=======-:··..
.··:-+*###*+=-:·. .·:-+*%@@@@@@%*+-:.. .·:=+*####*=-:. ..·:::::·. .:-+*#%%%%%##+=-:·..
..··:::::::··.... ..··::-==++++==-:·.. .·:--====-::·. ..····.. .·-+#%@@@@@@@%*=-·..
............. .....······.... ...······... .. ·-+#@@@@@@@@@@*=:.
·:+#@@@@@@@@@@%+:.
·:=*@@@@@@@@@@@*-·
.:=*%@@@@@@@@@@*=·
.·-+*%@@@@@@@@%*=:.
..·:=+*#%@@@@@%#+=:.
..::==++******+=-:·..
.........................·····............····:····.. ..··········..::-------=----:::··....
...··::::-----::::::::::::---------:::····::--=+++++=-:··. .·::-==++++==--::·:::::::::::::::::···...
..·::-==++++****+++++=====++++******++==----==+*#%%@@%#*+-:·. .·:-=*#%%@%%##*+=-::········::::---:::··...
.·::--===-:·.. ..:-=*%@@@@@@@@%*+-·. ..·-=+*###*+=-:·. ..·:--==--:·. .·-=+*#%%%%##*+=-:·..
.....····...........··:--=+******+=-:·. .·:--===--:·.. ..···:···. ·:=+#%@@@@@@%#*=:·.
..................··········.. ..·····... ..... :-+#@@@@@@@@@#+-·.
.-+#@@@@@@@@@@*=:.
.·+#@@@@@@@@@@#=:.
·:*%@@@@@@@@@#=:.
..:-#%@@@@@@@%*=:.
..--*##%%%##*=-:.
..::=======--:··..
....·····..... ......... ...·····.. ..··::::::····..··::···················...
..·:::-------::···......····::::···.. ..·::-===--:·. .·:-=+******+=-::·..... ....···::-----::··..
.·:-==++****+++=--:::···:::--====--:··.. .··-=+*###*+=-:·. ..:-+*%@@@@@@@%#+=:·.. ..·::-===+==--:··.
...······... ..·:-=*#@@@@@@@@@@#+=:·. .·:-=*#%%%%#*+=-::···:--=++***++=-:··.. ..·:-=+*##%%##*+=-:··.
......········::--=+********+=-·.. .·:-==++==--:··.....··:::----::·.. .··=*#%@@@@%#*+-:·.
...............····::::::···.. ..·······... .......... ·:+#@@@@@@@%#+-·.
:-#@@@@@@@@%*=:.
.:=%@@@@@@@@#+:.
..-=@@@@@@@%#+-·
..-=%%@@%%#+=:·.
..--+++++=--:·. ...
..··::::······.........···..
..·········... ..... ....·.... ..···:::::::··... . ...···::::---::::··.
..·:-=======-::·.. ...·····.. .·::---::·.. ..·-=+*******+=-:·. ..·:--=+++*+==-::·.
..·:-=**###**+=-::·.. ..··:::::·. .·:-++**+=-:·. .·:=*%@@@@@@@@%*=:·. .·:-=+*#####*+=-:·.
....······.···::-=*#@@@@@@@@@%#+-:·.....·:-+*#####*++=------==++**##***++=--::::::::--=++******+=-:·.. .....
..····:·····:::-==+********+=-··. ..·::-======--::·····:::--====---::···.......··::-*#####*+=-:·. ....
..·······.....····::::::::··.. ..········............·······.... ..--#%%@@%#*+-:. ...
..-=%@@@@@%*=-·. ..
..==@@@@@%#+=:. ...
.·==%@@%%#+=:·. ...
..--****+=-:·. ...·..
..··-----::··... ..····..
....················::::··.
..···:::::··... ........ ...····... ..··:::::::···.. ..·::--======----::··.
..·:-=+***++=-:·.. ..··:::··. .··:-----:··. .·:-=+*******+=:·. .·:-=+***#***+==--:··.
.·:-+*#@@@%#*+-:·. .··:---::·. .·-=+***+=-:·. .·-=*%@@@@@@@@#+-· .·:-+*#%@@@@%%#+=--:·..
..··:::::::·:::-=+*#%@@@@@@#*=-·. .··:-------:::::::---==++*******++++============++++===-:··. ..··:::··.
..··::::::···:::-==+*****+=-:·.. ..··::::::::·······:::---======----:::········::::-+++==-::·.. ..··:::··.
...······......····::::::··.. ....................···········..... ....--****+=-:·.. ..··::··.
..-=####*+=-:·. ..·····.
.·==%%%#*+=-·. ..·····.
..-=###**=-:·. ..·····.
..:-++++==-:·.. ..··::··.
.··-------::··......··:::··.
.··::-----:::·····::::::·.
...···::::··... ....·····... ..········... ...··::::::··.. .·::-==++===-----::::::·.
..·:-==+****=-:··.....··:-----::·. .·:-===+==-:··. ..·:-+******+=-:·. .·:-=+*###**++===---:::·.
.·-=+*#%@@@%#+-:·....··:-=++++=-:·. .:-=*#####*=-:·. .·:=+#%@@@@@@#*=:. .·-=*#%@@@%%#**+==--::··.
.··:-----::::::-==+*###**+=:·. ..........··::---===+++++++***++++++=====-------:::··... ..·:-=+++=-:·.
..··:::::·······::--====--:·.. ...........···:::----=======---:::············::---::···....··:-=+++=-:·.
...···...........·······.. .......·············.... ...::+++==-::··..··:--====-:·.
..--****+=-:··...··:--==--:·.
.·==###*+=-:··.....·::----:·.
.·==###*+=-:·... ..··::--::·.
..--****+=-:·... ...·::--:··.
..::+++++=-::·.....··:::::··.
··==+++==-::·....··:::::··.
...···::··.... .....······... ..··::::::···.. ...···:::···.. .·-=+++++=--:······:::::··.
.··:-==+**+=::··..··::--==--::·. ..·:-=+****++=:·.. ..·:=++****+=-:·. .·-=+*****+=-::··:::::::··.
.·:-=+*#%@@%*=-:····:-=+****+=-:·. .·:=+#%@@@@%#*=-·. .·:=+#%@@@@@#*=:·. .·:-+*#%%##*+=--:::::::::··.
.·::-----::::::-===++==-:··. ..··::---=========+++++++++===-----::::::::····....···:-=*###*+=-:·.
.········....···::::::··.. ..···:::---------------:::···......·······--::::···::-=+*#%#*+=-:·.
........ .......... ......·············..... ....:-++==--:::::-==*###*+=-:·.
··==##*++=-:::::-=+****+=-:·
··++%%#*+=-::··::-=+++++=-:.
··++@%#*+=-:·····:-==++==-·.
··++%%#*+=-:·....·:--===-:·.
··==###*+=:··.....·:--=--:·.
..-=##*++=:··.....·::----:·.
...····.. .......... ..··:::::::::·.. ...········... ..--***++=-:·.. ...·::--::·.
.··:-===-:··.....··:::::::··. ..·:=+*******+=-:·. .·-=++***+==:··. .··:-****+=-:·.. ..·::--::·.
.·:-=+*##*=-:··.··::-=++++=-:··.. ..·:-+*%@@@@@@@%*=:· ·:=*%@@@@@@#*=:·. .·:=+*****+=:··.....··:::::·.
.·:::-------===+++===-::··...... .·:--==++++====-------======------::::::::::::·······::--=+*#%%#*+=-:.
...··········:::::::···... ..··:::---------:::::::::·····.......·····:::·--::::::--=+*#%@%#*+=-:.
. .......... ....·············...... .......--+==------==+*#%@%#*+=-:.
··==**+==-----=+*#%@%#*+=-·.
::++%#*+=------=+*#%%#*+=-·.
::*+%#*+=--:::--=+*###*+=:·.
::*+%#*+=-::··::-=*##**+-:·.
::++%#*+-::····::=+***+=-:·.
··++#*+=-:··...·:-=++++=-:·.
.... ..·::::::::::·.. ..···:::···.... ....···==#*+=-:·.....·:-=++==-:..
.······.. ...····... ..·:=+*********=-·. .·:=+******+=-:·... ...·::--==**+=:··. ..··-====-:·..
.·::--::·.. ..··:::::::··.....·:-=+#@@@@@@@@@#+-·. .:=*%@@@@@@@%*=-·.. ..·:-++***++=:·.. ..·:-===-:·.
██████╗ ███████╗███╗ ██╗ ██████╗██╗ ██╗███╗ ███╗ █████╗ ██████╗ ██╗ ██╗███████╗
██╔══██╗██╔════╝████╗ ██║██╔════╝██║ ██║████╗ ████║██╔══██╗██╔══██╗██║ ██╔╝██╔════╝
██████╔╝█████╗ ██╔██╗ ██║██║ ███████║██╔████╔██║███████║██████╔╝█████╔╝ ███████╗
██╔══██╗██╔══╝ ██║╚██╗██║██║ ██╔══██║██║╚██╔╝██║██╔══██║██╔══██╗██╔═██╗ ╚════██║
██████╔╝███████╗██║ ╚████║╚██████╗██║ ██║██║ ╚═╝ ██║██║ ██║██║ ██║██║ ██╗███████║
╚═════╝ ╚══════╝╚═╝ ╚═══╝ ╚═════╝╚═╝ ╚═╝╚═╝ ╚═╝╚═╝ ╚═╝╚═╝ ╚═╝╚═╝ ╚═╝╚══════╝Compare the whole catalogue · Browse model reports
Choose a field to explore public problems and discussions.
Fields and counts refer to this catalog page.
Reasoning / Science
arc-challenge
ARC-Challenge is the subset of grade-school science questions that both a retrieval baseline and a word co-occurrence baseline answered incorrectly. It tests reasoning with knowledge rather than recall alone.
Paper: arXiv:1803.05457
Reasoning / Science
arc-easy
ARC-Easy is the subset of grade-school science questions that a retrieval baseline or a word co-occurrence baseline answered correctly. It is the easier counterpart of ARC-Challenge and serves as the baseline measure of …
Paper: arXiv:1803.05457
Science
critpt
CritPt evaluates scientific understanding and multi-step reasoning and computation on research-level physics problems. Its public dataset contains 70 challenges with problem descriptions and code templates.
Paper: arXiv:2509.26574
Code / Science
ds-1000
DS-1000 builds 1,000 data-science problems from real StackOverflow questions across seven libraries including NumPy, Pandas and Matplotlib. The problems are perturbed so that recalling the original answer does not solve …
Paper: arXiv:2211.11501
Science
frontierscience
FrontierScience evaluates the ability to solve expert-level scientific tasks. Its public data separates olympiad and research problems so that competition and research tasks can be examined independently.
Science
gpqa
GPQA is a set of graduate-level questions in biology, physics and chemistry written so that non-experts with web access still fail them. The Diamond subset is the most-used slice.
Paper: arXiv:2311.12022
Mathematics / Science / Multimodal
olympiadbench
OlympiadBench evaluates scientific reasoning on Olympiad-level mathematics and physics problems. Its official description lists 8,476 English and Chinese problems with separate text-only and multimodal settings.
Repository: Official repository