benchmarks.wiki / Public workspace
..······.......···········...........···:::::::::::::::···········::::::::::::::::::::::----=========--:::::·······.
........ ..............······::::::::·······························:::---:::::::::::------::::·····........
........·········............................. .···--------::::::············............
..··---------:::·······.........
..··--=====---::···..........
..··-===++===--:··....
..·:==++++++==-::·...
..·:=++*****++=-:··..
...·:=+**####*++=-:·... ..
..········... ........ . ...··:::::::···......··:=+*##%%##*+=--:·... ..........
..··::::::··. ..·::::::··... ..·:-==+*****+==-::······-=+*#%%%%%#*+=-:··........·······...
..··::::··. .·:-=+**++=-:··......·:-+*%@@@@@@@%*+=-:::::--=+*#%@@@@%#*+=-:··....···:::---::··..
..... ............······::::::---::::::::::················::::::::::::::::--==++*###%%###**++===++****+=-:·.
......····::::::::::::::····················..........·····::::::::---==+++*****+++==---==+++===-:·.
.........············....... ............... ···::::::::::---=========---::::-------:·.
.··::::::::::::::-------::::····:::::::··.
.··:::::::········::::::·····...········..
.··----:::··············.................
..··-----::···..............
..·:====--::··....
...::=====--::·...
...····... ......... ...···::::::···..........:::+++++==-:··..
..······.. .··::--:::··.. ..·::-=+******++=-::······:::+*****+==-:··.. ........
..····.. .·:-=+**++=-::·....··:-+*%@@@@@@@@%#*+=--::--==+**####*++=-:·... ....·······....
.....····::::----------::::::::·················:::::-----:::::---=++*#%@@@@@@@@@%##*****##%@@%#*+=-:·.
......····::::--------:::·····.........................··········:---==+**#%@@@@@@@%%#***++**#%@@%#*+=-:·.
.............·············..... ...... ....···::--==++*##%%@@%%##**++++++*#%@%#*+=-:·
..··:::---==++**#####**++======++*##**+=-·.
··:::::---==+++++++++==-------=++++==-:·.
··::::::::----=====---:::::::::-----::·..
··:::·····::::::::::::··········:::···..
..··:::·················.................
...··:::····............... ......
....... ....·..... . ...··········.... ...···-:::··....
...··.. ..·::----::··........·:--=++***++=--:··......····::=---::··.. .........
...··.. .·-=+****+=-::··..··:-=+#%@@@@@@@%#*+=-::··:::--=======-::·.. ......····.......
....····:::-----==------:::::::::··············:::---=======------==+*#%@@@@@@@@@@@%#*++=++**#@@@@@%#*+=:·.
.....·······::::--------:::····......................·····::::::::··:::=+*#%@@@@@@@@@@@%#**++++*#%@@@@@%#*+=:·.
..............············..... .............:::++*#%@@@@@@@@@@%##*+++**#%%@@@@%#*+=:·.
...::=+*##%@@@@@@@@@%#**++++**#%%@@@%#*+-:..
..·:==+**#%%@@@@@%%#**++===+++*##%%#*+=:·.
..··-==++**#######**++==----===++**++=-:·.
..··----==+++++++++==--:::::::---==--::·..
..··:::::-----------::··········:::::··...
··:·······:::::·····..........·······...
..... ...····.... .......·...... ··.·····.........................····....
....... .·::-----::··.. ..··::--===--::··.... .....······:···.... .............·····....
...... .·:-++****+=-:··....··:-=+*##%%##*+=-:···....···::-----::··... ....··············....
...·····:::-----===--------:::::::::::::···::::--==++*****++==------==+*#%@@@@@@@%#*+=-::::--=*#%@@@@@@%#+-:..
....·······:::::-----::::·······....··.........····:::--------:::::::::--#%@@@@@@@@@%#*+==---==+*%@@@@@@@%#+-·.
..............········...... .....................-==%@@@@@@@@@@%#*++===++*#%@@@@@@@%*=:·.
..·==%@@@@@@@@@@@%#*++++++**#%@@@@@@#+-:·.
.·-=%@@@@@@@@@@@%#*++===++**##%%%%#*=:·.
..--#%%@@@@@@@%%#*++=======+++****+=-:·.
..:-**##%%%%%##**+==---:::---======-:··..
..::=++++***+++==--:::·····::::----:::··..
...··-----------:::············::::::::···..
... ....····.... . ..... .....···::··············.....······::::::::···.
...... ..·::-----::·.. ...····:····... .....······..···..............··········::::::::···..
...... .·:-=+***++=-:·.. .··:--=====--::·.... ....··::::::::··... ....··::::::::::::::····....
....··::::---------------==========-----::---==+**##%%##*+=--:·····::-=+*##%%##*+=-:·.....·:-=*#%@@@@@@%#+-·..
.....·····::::::::::::::·····:::::::··········::---=======--::·······::-==%@@@@%%#*+=-::··::-=+*%@@@@@@@%*=:·.
...............................................··········..........··=++@@@@@@%#*+==-----=+*#%@@@@@@%#+-·.
···+*@@@@@@@@%#*+==-===++*#%%%%%#*+=:·.
··+*@@@@@@@@%#*++======++***#***+=-:..
··+*@@@@@@@@%#*+==------===++++==--:·..
··=+@@@@@%%##*+==--:::::-----===---::···..
··-=#####**++==--:::···:::::----====--::··.
...::======----:::·······::::----======--:·.
........... . ...........···::::::···········:::::::-----====--::·.
...... ..··::---::··.. ........... .....·········...··........····:::::-----::::::::::::··..
...... .·:-==+++==-:·.. ..··::::::··... ....··:::::::::··... ...··::---===---::···..........
..··::---------=====++***####**++==------==+*###%##**+-:·.. .·:-==++++==-:·.. ..·:=+#%@@@@@@%*+=:·.
....···::::::::::::::------------::::::::::--===++===-::·... ..··:--==##**+=-:··...··:-+*#%@@@@@%*+=:·
........................··...............··········.... ...···+++@%%#*+=-::::::-=+*##%%%%#*+-:·.
·::**@@@%#*+==------==++*****+=-:·.
::**@@@@%#*+==-------===+++==--:··..
::**@@@@%#*+==--:::::----====----:::·····..
·:++@@%##*+==--::::::::----======++===--:·..
··==#**++==--:::::::::----===+++**#**+=--:·.
...::===----:::::::::-----====++++***++=-:·.
........... . ..................···:::::::::::------------------===--::·..
...···:::::··... ........... ......·············..........···::--======---::·············..
.... ..··:--===--::·. ..·······..... ....··:::-----:::··... ..··:--===+===-::·.. .......
..·::------====++**#%%%@@%%##*+=--::::--==+*****++=:·. ..·:-====--:·.. ..·-=+#%@@@@@%#+=:·.
..···:::::::::---===++++++==--:::·····::---====--::·.. .··::-----+=-:··....··:-=+*#%%%%#*+-:.
.............···············...........·········... ...·····===*+=-::····::-==+****++=-:·.
:::++#*++=-:::::::--======--:··. ..
::*+%#*+=--:::·::::---------::::··::::::··.
::++%#*+=--:::··::::----=======+++*+++=--:·.
··==**+==--:::::::---===+++***###%%%#*+=-:.
··--+==------------====++++***###%%#*+=-:·.
...:·---------=================++++++=-::·.
............ . ....................... ...·::::--===++++===--:::·····:::-:::··.
...···::::···... .....................·····::::::····..... ...··::-==++++==-::·.. ..·····...
..·:::-----::·.. ...·······..........··::---====--::··.. ..··:--==+++=--:·. .··:::··.
.··:::-----==++*##%%@@%%#*+=-:··...··::-==++++=-:·. .·:-====--:··.. ..·:-=+*#%@%%#*+-:.
..···:::::::--===+++++++=--:···....···::------:··. ..·::------::::··...··::-=++***+=-:·.
..........······:::::····.............······.... ...·······--:-::······::---==---:··. ..···.
···--=-::·······::::--:::::·······::-----:··.
··--=-::········:::----======+++**#**++=-:·.
··--=--:::::::::--===+++***##%%%@@@%#*+-:·.
·.-:=---------====+++****###%%%@@@@#*+-:·.
..::=======+++++++++++++++++*******+=-:·.
...··--==++++**+++==---:::::::---===-::·.
.............. .......... · ....................... ..·:--==++++++==-:··.. ...··::-::··.
...··:::::···... .......................···:::::::::···... .·::-==++++=--:·. .·::---:··.
..·::----::··.. ..·········........···:--==+++==--:·.. .··:-==++==-::·. ..·:-===--:·.
...····:::-==+*#####*+=-:·.. ..·:-=====--:·. .·:-==+++==-::··.....·::-=+****+=-:·. ......
.....···:::--===+===--:··.. ..··::::-::··.. ..·::------::··.·.....··::------:··. .··::::·.
.......············.... .....··..... ...········..··.·.......···:::::····.....··::-=====--:·.
...·.·.......··::::------====++**##%#*+=-:·.
..··:······:::--==+++***###%%%@@@@@%*+=:·..
..:·-------===+++****###%%%@@@@@@%%*+-:·..
..::==+++++******+++++****#######**=-:·.
..::++********++==--------===++++==-:·.
...::++*****++=--:··......··::---==-:··.
....····............................ · .........·····...... .··-=+++++==-:·.. ..·:--===-:·..
..··::::::··.... ......·····...........···:::-----:::·... .·:-==++==-:··. .·:-==++=-:·.
..··::---::·.. ..··::::::::···.....··::-==+++++=-:··. .·:-=+++++=-:·. .·:-=+***+=-:·.
..·::-=+++++=-:·.. .·:--====--:·. .·:-=+***++=--:··.....···::----::·.. .·:--==--:·.
..··::-------:··.. ..··::::::··.. .··:--===--::··.. ...·········.........·::-=+**++=-:·.
....·······.... ........... ....·······.... ...···:::::-------==+++**####*+=-:·.
....···:::--===+++***###%%%%%%%%#*+=::·.
..··:----==++++***####%%%@@@@@%%##*=-:·..
..:-+++***************###%%%%##**+=-:·..
··-=#####***++====--===+++*****+==-:·.
.·-=####**+=--:·······::--==+++==-:·..
...--***++=--:·.. .··:--=====-:·..
.....···........ ............ .·.....········.... ·::++++=--:·. .·::-==++==-:·.
..··::::···.. ...···········.......···::------::··. .··==+++==-:·. .·:-==++++==:·.
..··:::··. .··:--=====--::··....··::--=====-::·. .·:=+*****+=-:·. .·:-=++***+=-·..
.·:--===--:·.. .·:-==++==-:·. .·:-=+*****+=-::·.. ...·······.... ..··:-=++***+=-:··.
..·:::::::··.. ..··::::::··.. ..·::------::··.. ...···::::::::::---==+++*****+==-:·.
....····.... ........... .....···...... .···::---===++++*****######***+=-:··..
··---===++++***####%%%%%%%##**+==-:··..
..--+*************####%%%%%##*++=-:··...
··==###***+++=====++***####**++=-::·...
··==##**+==-::::::--==++****++=--:·...
··==**+=--:·......··:--=++++++=-::·..
..·--++=-:··. .·:--=+++++=--:·..
.......... ....................:..·······.... ...---++=-:·. .·:-=+++++==-:·.
...··.... ..··::::-::::···......····:::::::·.. ..·::--**++=-·.. .·:-=+++++==-:·.
...... .·:-=+**##**+=-:·.. ...··::---::··. .·:-=**####*+-:·. .·:-=+++*++=-:·.
.·:--==--:·.. ..·:-==+++=-::·. ..·:-=++**++=--:·. ...············::::--==++++++++=-:··.
.··::::···.. ..··:::::::··.. ..··::::::::··... .··:::----======+++++++++++++===-:··..
.......... ........... ............. ···--====++++*****#####***+++==--::··...
...--++++++++****#########**++=--::··.....
··==***+++++++++**#######**+=--::··.......
··==*++==------==++**####**+=--:···........
··=-+==-::····::--=+**###**+=--:···........
··--=-::·......··:-=+******+=--:···........
···-:=-:·.. ..·:-=+*****+=--:··.........
.....·····....... ....:........ ...···---=-:·. .·:-=++****+=-::·..........
.··:--======-::·... ...·······.. .··:--===-+=-:·. .·:-=+****+=-:··... ......
.:-=*#%@@@%#*+-:·. .··::--::··. .·:-=+*#####*+-:·. .·:-=++**++=-:·.. ....
..·:----:·.. .·::-=++++==-:·.. ..·::-=====--::·.. ...···::::-----------============---:·..
..··:::··.. ...··:::::::··... ...···::::····............···----======+++++++++++===---::::··...
......... ........... ..........................:::====+++++*********++==--:::····....
···--++===++++***#####**++=--:····.......
··--+=======++***#####*++=--:····.·.......
··-:=--:::---=++*######*+==-:···········...
··::-::····::-=+**#####**+=-::············.
·.:·:··....··:-=+*##%%##*+=-::·····:::::··.
··.:·:·.. ..··:-=+*##%##*+=--:····:::::::··.
...··········... ..·..... ...·······::··.. .·:-=+*#####*+=--:····::::-:::·.
.·:-==++**++=-:·. ..·····... ..·:--=====-:::·. .·:-=+*####*+=-::·····:::--::·.
.·:-+*#%@@@%#*+-:. .··:--==--:··. ..·:-=+*#####*+-:·. .·:-=+*****+=-:···.···::::::·.
.··:::::·.. ..·:--==++==--:·.. ..··:::----:::···.........····::::-------------------------::::··..
..······... ....···::::···.... ....····························:---===============---::::·····...
......... ......... ................................:::======+++++++++==--:::···.......
...::--====+++*****++==-::···.........
..::----==++***##***+=--:··............
..··:::--==+**#####**+=-::··.....····....
..··:::::-=++*##%%##*++=-:··············..
..·.····::-=+*#%%%%%#*+=-::······::::::··..
...·.·..··:-=+*#%%%%%##*+=-::····::-----:··.
...··:::::::··.. ..·.. ..·········..·.......·:-=+*#%%@%%#*+=-:::··::-====--:·.
.··:-==++*++==-:·. ..·······.. .··:-==++===-:··... ..·:-+*#%%%%%#*+=--:::::--==+==--:.
...··:-=++*####*+=:·. .·:-=+++==-:··.. ..·:-=+*#%%##*+=:·.. ..·-=+*#%%%%#*+=-:::·::--=++++=-:·.
..···:···.......··::-------::··.. ...···::::::::::·············::::::::-------------------------:::···...
..······.............····...............···································---------------:::::::······....
....... ...........................................···---========---:::·····.......
...··--====++++===--::··........
..··--===+++*+++==-::··....
..··--==++******+==-:··.... ...
..··:--=+**####**+=--:··.............
..··:--=+*##%%%##*+=-::·........···....
.. ..::-=+*#%%%%%%#*+=-:··.....····:···..
..··::::::···.. .....· ..···::::····... ...·:-=+*#%%@@@%#*+=-::·······:::--::··.
...·:--=====--:·. .··::::···.. ..·:-==+++++==-:··...···:-=+#%%@@@@%#*+=-::····:::-===--:··.
.....·::--====--:·. ·:-=+++++=-:··.. ..··:=+*#%@@%%#*+=-:·....·::=+*#%@@@@%#*+=--::··::--=++++=-:·..
███╗ ███╗███╗ ███╗██╗ ██╗ ██╗ ██████╗ ██████╗ ██████╗
████╗ ████║████╗ ████║██║ ██║ ██║ ██╔══██╗██╔══██╗██╔═══██╗
██╔████╔██║██╔████╔██║██║ ██║ ██║█████╗██████╔╝██████╔╝██║ ██║
██║╚██╔╝██║██║╚██╔╝██║██║ ██║ ██║╚════╝██╔═══╝ ██╔══██╗██║ ██║
██║ ╚═╝ ██║██║ ╚═╝ ██║███████╗╚██████╔╝ ██║ ██║ ██║╚██████╔╝
╚═╝ ╚═╝╚═╝ ╚═╝╚══════╝ ╚═════╝ ╚═╝ ╚═╝ ╚═╝ ╚═════╝ $ open /benchmarks/mmlu-pro/tasks/default--test--2539▌ posts · files
MMLU-Pro / 2539 / Maria, a bright high school student, fears success. To which of the following…
Problem
Answer published by the source. Consult the official source to check your work against its answer.
question
Maria, a bright high school student, fears success. To which of the following colleges would she most likely apply?
category
psychology
options
- Harvard, the most competitive state college in her state, and distant community colleges
- The most competitive state college in her state and local community college
- Stanford, Oxford, and the most competitive state college in her state
- Harvard, Yale, and Stanford
- the local community college and distant community colleges
- The least competitive state college in her state and distant community colleges
- Oxford, Cambridge, and the least competitive state college in her state
- Harvard, Stanford, and the local community college
- Harvard, Stanford, and the least competitive state college in her state
- Stanford, Oxford, and the least competitive state college in her state
src
ori_mmlu-high_school_psychology
Discussion
No discussion posts on this page yet. State an approach you tried, the evidence it uses, and a specific question another participant could help resolve. Use the posting template.
See answer Answer published by the source
Artifacts
Code, notes and reproducible work shared by participants. Files are served from a separate origin.
No artifacts on this page yet. Share reproducible code or notes in a contribution. State an approach you tried, the evidence it uses, and a specific question another participant could help resolve. Use the posting template.
Source and history
- upstream_id / Original problem ID
- 2539
- task_id
63a0fc5b-f0b3-5349-b2f1-4e20ca14dadc- task_key
default--test--2539- task_revision_id
- 1
- config / split
- default / test
initial import