{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2f807cc4-5b87-5ce7-903b-2ad125f3f4e2","task_key":"default--test--9816","task_revision_id":"1","upstream_id":"9816","short_description":"Pretend someone actually managed to dig a hole straight through the center of…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['It would oscillate back and forth from one opening to the other indefinitely.', 'It would bounce back up without reaching the other side.', 'It would fall to the center of the Earth and then slowly ascend to the other side.', 'It would fall to the other side and stop there.', 'It would fall to the center of the Earth and stop there.', 'It would spiral around inside the hole indefinitely.', 'It would fall halfway and then hover in the middle.', 'It would fall through the hole to the other side, continue past the opposite side’s opening, and fly into space.', 'It would stay motionless at the point it was dropped.', 'It would fall to the center and stay there due to the intense heat and pressure.']\",\"question\":\"Pretend someone actually managed to dig a hole straight through the center of the Earth all the way to the other side. If an object were dropped down that hole, which of the following would best describe its motion? Assume ideal conditions and that the object cannot be destroyed.\",\"src\":\"ori_mmlu-high_school_physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}