{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"35553e5a-df12-555d-bce7-f9577af93a74","task_key":"default--test--9437","task_revision_id":"1","upstream_id":"9437","short_description":"An inflated balloon with a heavy rock tied to it is dropped into a deep lake. As…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['is not affected by the depth of the lake.', 'Need more information.', 'decreases initially, then increases.', 'depends on the size of the balloon.', 'remains largely unchanged.', 'decreases.', 'increases initially, then decreases.', 'increases.', 'depends on the size of the rock.', 'oscillates between increasing and decreasing.']\",\"question\":\"An inflated balloon with a heavy rock tied to it is dropped into a deep lake. As the balloon sinks deeper and deeper the buoyant force acting on it\",\"src\":\"ori_mmlu-conceptual_physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}