{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6048e178-e3a8-5afb-84d8-78c3a65e0617","task_key":"default--test--9542","task_revision_id":"1","upstream_id":"9542","short_description":"In a circuit of two lamps in parallel, if the current in one lamp is 2 A, the…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['about 1 A', 'Depends on the resistance of the lamp', 'Depends on the power rating of the lamp', '0 A', '2 A', 'Depends on the voltage of the battery', 'About 0.5 A', 'Not enough information to say', 'Depends on which lamp is closer to the battery', '4 A']\",\"question\":\"In a circuit of two lamps in parallel, if the current in one lamp is 2 A, the current in the other lamp is\",\"src\":\"ori_mmlu-conceptual_physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}