{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7060319b-98e6-5deb-a64a-f7a0c5d47886","task_key":"default--test--10168","task_revision_id":"1","upstream_id":"10168","short_description":"According to Hooke’s law, if you hang by a tree branch and note how much it…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['three times the bend.', 'twice the bend.', 'four times the bend if the branch doesn’t break.', 'no bend at all.', 'a bend that is unpredictable.', 'half the bend.', 'a bend that is proportional to the square root of the weight.', '4 times the bend.', 'a bend that decreases with increased weight.', 'the same bend if the branch doesn’t break.']\",\"question\":\"According to Hooke’s law, if you hang by a tree branch and note how much it bends, then hanging with twice the weight produces\",\"src\":\"ori_mmlu-conceptual_physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}