{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9bdc675a-8ffc-593a-862c-87557c475024","task_key":"default--test--9937","task_revision_id":"1","upstream_id":"9937","short_description":"Compared with the current in an LED lamp the current in the connecting wire is","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['The current in the wire is half as much.', 'Depends on the voltage.', 'Need more information.', 'The current in the wire is twice as much.', 'Depends on the type of LED lamp.', 'the same.', 'The current in the wire is not related to the current in the lamp.', 'less.', 'The current in the wire fluctuates.', 'more.']\",\"question\":\"Compared with the current in an LED lamp the current in the connecting wire is\",\"src\":\"ori_mmlu-conceptual_physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}