{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"924004c1-0b77-52f8-bb63-1415fd31ea8f","task_key":"default--test--9788","task_revision_id":"1","upstream_id":"9788","short_description":"When 2.00 lb of brass at 2120F is dropped into 5.00 lb of water at 35 .00F the…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['0.095 Btu/lb°F', '0.110 Btu/lb°F', '0.091 Btu/lbF0', '0.082 Btu/lbF0', '0.065 Btu/lb°F', '0.100 Btu/lbF0', '0.075 Btu/lb°F', '0.120 Btu/lb°F', '0.105 Btu/lb°F', '0.085 Btu/lbF0']\",\"question\":\"When 2.00 lb of brass at 2120F is dropped into 5.00 lb of water at 35 .00F the resulting temperature is 41.2 F. Find the specific heat of the brass. (Neglect the effect of the container.)\",\"src\":\"stemez-Physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}