{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4a6b2485-5a8e-586c-8b73-c8309c6df524","task_key":"default--test--9554","task_revision_id":"1","upstream_id":"9554","short_description":"A 'fishbowl' of height 4r/3 is formed by removing the top third of a sphere of…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['20.5', '24.0', '16.4', '22.3', '18.25', '23.1', '21.7', '19.6', '17.0', '15.75']\",\"question\":\"A 'fishbowl' of height 4r/3 is formed by removing the top third of a sphere of radius r=6. The fishbowl is fixed in sand so that its rim is parallel with the ground. A small marble of mass m rests at the bottom of the fishbowl. Assuming all surfaces are frictionless and ignoring air resistance, find the maximum initial velocity that could be given to the marble for it to land back in the fishbowl with g=9.8.\",\"src\":\"theoremQA-Physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}