{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cf39e984-8878-57d3-babe-fd5048b3b17e","task_key":"default--test--7508","task_revision_id":"1","upstream_id":"7508","short_description":"A student eats 3 slices of pizza while studying for his Economics exam. The…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['The total utility this student received from eating pizza is less than 20 utils.', 'The student would not eat any more pizza.', 'The total utility this student received from eating pizza is 20 utils.', 'The marginal utility of each slice of pizza is equal.', 'The student should have stopped eating pizza after 2 slices.', 'The marginal utility of the 4th slice of pizza will be 0.', 'The student should have only eaten one slice of pizza.', 'The total utility this student received from eating pizza is 30 utils.', 'The marginal utility of the 4th slice of pizza will be negative.', 'The student should have stopped eating pizza after the first slice.']\",\"question\":\"A student eats 3 slices of pizza while studying for his Economics exam. The marginal utility of the first slice of pizza is 10 utils, the second slice is 7 utils, and the third slice is 3 utils. Which of the statements below holds true with the above data?\",\"src\":\"ori_mmlu-high_school_microeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}