Discussion
0 posts
No posts in this thread yet.
benchmarks.wiki / Public workspace
.·:-+*#%@@@@@@@@@@@@@@@@@@@@@@@@%#+=-:·.. ....... ..··:--=====-::·. .·:-=+++=-:·..
..·::-=============++++*****++==-::··.. ....·····.... ...··::------::··.. ..·-=+==-:·..
................·············.... ......····....... .....··········... .·-===--:·..
..:-----:·..
..:----::·..
..::--::··..
..::::::··..
.·::::::··..
..··---:::··..
.....·········..... ....................·······:::::·················..........·:----::··..
...·······::::--===++++++==--:··... ..................··::--===++++++++======-----------------::::==--::··..
.··:--==+=++++**#%%@@@@@%%#*+=-:·.. ..... ..·::-=+*##%%%%%%##****+++++****************+++=--:··..
..·:-=+*******+++++***#%%@@@@@@@@@%#*++=--::::::---======--::·.. ..··::-----::·.. .·:=+*%%#*+=-:.
..··:::-:::::·····::::--=========--:::·······:::---=====---::··.. ...···:::::··... ..·:+###*+=-:·
....... ............... .....····::::::::···..... ............ .·=****+=-:·
.·-=+++==-:.
.:-====-:·.
.·:----::·.
.·::::··..
.······..
.·····...
..........····:::::::::····.... ..............······::::::::::::::········...... ..·····...
...···:::::::--===++*****+++=--::··.......................···::--=++***********++++===----:::············:::···..
.·::--========+++**###%%##**+=--:··................... ...··:-=+*#%@@@@@@@@@@@@@%###**++++===============-::·..
..··::-:::····....···:--=+**#####**++===--===++*##%%@@@%%##*+=-:·.. ..··:-==++**++==-:··.. ..·:-=+***++=-·.
...······.... ...···::::::::·····.····::--==+++***++==--:··.. ...··::------::··... ..·:=****+=-:·.
...... ....···::::::::···.... .....···..... ··=+***+=-:·.
.·-+++++=-:·.
.·-======-:·.
..:------:·.
.·::::::··.
.·······..
.·····...
..............·····:::::·····........................····················............... ..·····...
..··::::::·····:::---=======---:::·····.·············:::::::---===+++++++=====-----::::::·····........···::::··..
.·:-==++++==--------====++====--::···········::::::::::::::---==++*##%%@@@@@@%%%##***++===---:::::::::---=====-:··..
..···::···... ..··::--===+====---::::--=++*#%@@@@@@@@@@@%#*+=-::···:::-==+******+==-:·. ..·:::::··.
..··::::···...........····:::::····.........··::-=++*********+=--:···...····::--======-::··.. .·::::::·.
....·....... ........... ....···::::::::····.... .....··········... ··::--::··.
.·::----:··.
..·::---::·..
..::----::·.
.·:---:::·..
··:--:::··.
..··---::··..
.......... .......········........... .............····....................................··----:··..
.··:------:::·······::::------:::::························::::::------::::::::::::::::::::··············:==--::·..
.·:-=+*####*+===----===========---:::::::::::-----------:::-----====+++++++++++=========---:::········::--=====-:··.
.·::-====----::::::---=++*******++==--::::--=+*#%@@@@@@@@@@@@@@%#+=-:··....···::::::::··.. ......
..·::--------:::::::::----=====---::··.......··:-=+*************++=-:··......····:::::···.. ......
...········..........··········.... ...··::::::::::::::··..... ............... ......
..····..
.····::···.
..··-----::··.
..::=+===-::·.
..--+*++=-:··.
..:-+++==-:·..
...·····...... ................ ....··············...···············..........::===--:··.
.··:--====---::::::::::::::::::::·····...... ...··:::---------------------------:::···.......··:--:::·..
·:-=+*##%#**+++++++++++++++++===--::::······..............···::--====+++++++++++*****+++==-::··... ...··::::··..
..·-==+++*+++++++++++**##%%@@@@@%%#*++==-----=++*%@@@@@@@@@@@@%*+-:·. ..······...
.··:--====------------==++++**+++==--:::·····::--=+************+=-:·. .....··....
...········...........······:::·····...... .....··:::::::::::::··... ....... .
....··...
··:::::::··.
..::++++==-:·.
··--*#**+=-:·.
··--***+==-:·.
..::===--::·..
....···...... ...···::::::::::::::·········:::······..... ..······...
.··::----:::::·············........ ..··:--==++*****+++++++++++++++++===-::··.. .......
..·:--=++================---:::···.... ..··:-==+***############%%%%%%%%##*+=-:·.. ......
.·:-=++++++++========++**#%%@@@@@%%#*+==-::···:::-=+**#%%%%%##*+=-:·.. ..··::-------::·..
..·::-======---:::::::::--==+++*****++=--::···...···:--==+++++++=--:·.. ..···:::::···...
...···::::····............····::::::::···..... .....···:::::::···... ........... ..........
. ..:::::::··.
..::=====--:·..
..::++++==-::·.
..::=====--:·.
..··:-::::··..
..........
........ .......... ....···:::::::::::::::::::::::::::::::···...
...·······.......... .......······...........··::--==++********++++++++*********+==-::·.. ....
...····················..........·········.......···::-==++**#######******###%%%@@@@%%#*+=-:··.. ..······..
..·:-=+******++=--:::···:::--==++****++==-::·......··:-=+**#%%@@%%%#**+=--:···.....··::-==+***##**++=--:···.....
..··:-=+**+++==--::····..····::---=====--::··... ...·::--=++++++++==-::··... ...··:::--------::···.···......
...···::::::····..... ......··········... ...····::::::···.... .................········..
..·:·::····.
..·::::::··.
..·::::···..
..·······...
..·····....
..··.......
.......... ......·········.... .....····················:::::::::::···.... ..······...
.......···...........····::---------:::···......···:::-------------:-----===+++++++==--::···.....::::::··..
.....·················:::--===+++++==---:::····:::---=======---:::::::::--==++***+++==-:::·····::--==--::·.
.··:-=+*#####**+=-::·... ..··::--========--------==+**#%@@@@@@@@@@@@@%#**+=--:::·::::-==+*##%%%%##*++=-::··.....
.··:-=++**+++==-::·... ...··::::::::::·········::--=++************++=--::·········:::--------::··..·.......
..···:::::::···... .............. ....···::::::::::::::···.......................
...
......
..··........
..··::::····..
..::---:::··..
..::==---::··.
....·········....... ........···...... ....... .....·············.....::===---::·.
..··::--------:::·········::::------::···... .....·····..... ...···::::--::::·········-===---:·.
..·::--==+++++===----------===+++++===--::····..·····:::::::···.. ..···:::::····.....··::--==--:·.
.··:-=+*#####**+=--:··......··:--=+**##########*******####%%@@@@@@@@@@@@@@@@%%#*++=======++***##***+=--:·..
...·:-=======--::··.. ..··::--==========---------==+++*****************++==---------------::··..
...·········.... ....·············.......····::::::::::::::::::::·····..............
. ..·........
..:·--:::···..
··--++==--::·.
··--**++==-::·.
..--+*+++==-:·.
...··········... ....... ................::=++===--:·.
..·::--==+++===-::··.. ....······.... ........... ....·········.......·:------::·.
..·:--=+*###%###*+==-:···.....····:::::::···.........···::---==---::··.. ....···::::···... ...·::::::··.
..·:-+*#%%@@@@%%##*++=========+++****####******++++======++**#%@@@@@@@@@@@@@@@@@@@@%%#####%%%%%##*+=-::·.
..·:--==+++++===--:::·····:::---=====+++=====----::::::---==+*******************+++===========--::·..
...···········..............······::::·······.........····:::::::::::::::::················..... ...
..·.·...
..:·:::···...
..::===--::·..
··--+++==--:·.
..--***++==-:·.
..::+++++==-:·.
....····..... .................··-======-:·.
..··::-----:::·.. .... ..··::::::::::···········:::::::::::···......::------:·.
..··:-==++++++==-:··.. ........ .·::-=+*#######**++========+++++++++=--::·.......··::::::·.
..·:-+*%@@@@@@@@@@@@%#**+===-----------------===-----:::::::::--=+*#%@@@@@@@@@@@@@@@%%%%%%%%%@@@@@%%#*++=-:·.
.··:==+*********++==--:::::····:::::::--------::::····...····:--==+********+++==---------========--::·:·..
..··:::::::::::····..............············...... .....···········..... .............:·:··..
..··::··..
..··:::··...
..··--:::··..
..··----::··..
..··-----:::·.
..··------::·..
...·······.... ......····.............··········......··-------:··.
..··::----:::··.. ..·::--===++=====------=====+++++===--:::····:::---:::·.
..·:--=====--::··... ............ .··:-+*##%@@@@@%%###*****###%%@@@@@@%%#*++=-:::·:::::::::·.
..·:-=+*%%%@@@%%%##*++==--:::············:::----==----::··... ..··:--=+++**+++==----:---==+**#%%@@@@%%##*=-:·..
..··:-=++++**++++==--:::···...........····::::::::::··.... ...···::::::···.... ....··::--======-*+=-:·..
...···:::::::::····..... ......······...... ....... ........-:==-::·..
..::--::··..
.··:::···...
..······...
..······...
..·····....
..···.......
..····:::::····.... ....···········....··············..... ..··........
.·::--=====---::···.......·········.... ...··:--===++++=====-----====+++++++++==-::··..··........
..·:--==++===--::::··::::--===++++===---::::---=++*#%%@@@@@@%%##********##%@@@@@@@@@%%#*==-:····.......
..·::-=++++++=====--------:::::··········::--========--:·.. ...···::····... ..·:-=+*#%%@@@@@@#*=-:·.
...··::::-------::::::·····.... ....···::::::::··... ....·······... ..·::--==+++@%#*=-:·.
.......········....... ............. ........... ....··==##*+=-:·..
..--**++=-:·..
..::=+==--:·..
.··----:::·..
..::::::··..
..········..
............
...····:::::·····.........··········..... ......·······........................ . .........
..··::--====----:::::::::---====+++===---::::::::---====+++====---:::::::::----------::··... . ....
.··::--===----::::::::-==+*#%%@@@@@@@%%########%%@@@@@@@@@@@%#**++=======++**######*+==-:·..
..·:--=++*****+++++++++++++===--::···......···:::----::··.. ...·:::---:::··.. ..·:-=+*##%%@%#*=-:·.
...··::::--------------:::···.. ............ ...··:::----:::··... ..··::-==++@@%*+-:·.
..........·······..... ....·········..... ....··=+%@%*+=-·..
··==%%#*+=-:·.
.·-=#%#*+=-:·.
..--*#**+=-:·.
..:-+**+==-:·.
..::=++==--:·.
..·:===---::·.
....·········........·····:::::::::·····.........········....... ...............··----:::··.
...···:::::::::·········::--===+++++++===---------==========---::···......·····::::::·······:::::···..
......... ....··:::----:::··........··:--=+**#####****++++***##%%@@@@@@@%#**+=-::······:::--======---:::·······....
.·:-=+#%@@@@@@@@%%%%%%%%%%%%##*+=--:·.. ..... .··:--======--:·.. ..·:-==+++=-:·.
..·::-===========++++++++==--::·.. ..··::--=====---::·... ..··::+++=-:·..
.........···············... ....············.... ..::+*+=-:··.
..:-+*+==-:·.
..:-+*++=-:·.
..:-+*++=-:·..
..--**++=-:··.
..--**++=-::·.
..--**++=-::·.
............... .......·············.........··········............................--+*+==-::·.
...............··:::::-::::··.... ...····::::::::·······::::----=====---:::··········::::------------+++=--:··.
.···:::::::::---==+++++++=-::·.. ...···::::·········::--=++**#####**+==-::······::--==+++******+++++=--::·..
███████╗██████╗ ██████╗ ███╗ ██╗████████╗██╗███████╗██████╗ ███╗ ███╗ █████╗ ████████╗██╗ ██╗
██╔════╝██╔══██╗██╔═══██╗████╗ ██║╚══██╔══╝██║██╔════╝██╔══██╗████╗ ████║██╔══██╗╚══██╔══╝██║ ██║
█████╗ ██████╔╝██║ ██║██╔██╗ ██║ ██║ ██║█████╗ ██████╔╝██╔████╔██║███████║ ██║ ███████║
██╔══╝ ██╔══██╗██║ ██║██║╚██╗██║ ██║ ██║██╔══╝ ██╔══██╗██║╚██╔╝██║██╔══██║ ██║ ██╔══██║
██║ ██║ ██║╚██████╔╝██║ ╚████║ ██║ ██║███████╗██║ ██║██║ ╚═╝ ██║██║ ██║ ██║ ██║ ██║
╚═╝ ╚═╝ ╚═╝ ╚═════╝ ╚═╝ ╚═══╝ ╚═╝ ╚═╝╚══════╝╚═╝ ╚═╝╚═╝ ╚═╝╚═╝ ╚═╝ ╚═╝ ╚═╝ ╚═╝$ open /benchmarks/frontiermath▌ready
FrontierMath
Mathematics
FrontierMath evaluates advanced mathematical reasoning using problems written by experts. This catalog links to the official introduction and public examples while excluding private evaluation problems.
frontiermath
History
Epoch AI introduced FrontierMath in 2024 to evaluate advanced mathematical reasoning with problems written by expert mathematicians. The project addressed the need for difficult mathematical evaluations beyond familiar competition-style questions. Its private evaluation problems are distinct from the public examples used to illustrate the benchmark.
How it was built
FrontierMath uses expert-written mathematical problems rather than simply collecting an existing public examination. The evaluation includes private problems, while public examples provide a limited view of the task format and difficulty. This catalogue links to the official introduction and examples and does not reproduce the private evaluation set.
At a glance
- Answer access
- This catalog excludes private evaluation problems and provides official introduction and public-example links only.
What a problem looks like
Advanced mathematical problems written by experts.
Why it is hard
Requires advanced mathematical reasoning on expert-written problems.
Limitations and criticism
Success on advanced mathematical problems does not by itself measure the broader process of mathematical research, such as choosing worthwhile questions or developing a reusable theory. The private evaluation set also limits independent inspection of its coverage and individual outcomes. Public examples cannot establish how representative a reported score is of other mathematical domains.
Reference links
- Site
Related benchmarks
Discussion
1 thread and 0 posts on this page. Anyone may reply; no account is required.
Contribute
Share a reproduction, report an evaluation issue, or suggest a correction to this benchmark overview.
Public problem summary
Catalogue metadata only; public problem statistics are not available.
- Public problems
- —
- Formats
- —
- Problems with images
- —
- Answer availability
- —
- Configs
- —
- Splits
- —
- Awaiting a first discussion
- —
- Problems with a discussion
- —
- Public contributions
- —