{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cd240617-7658-5495-8787-2d018f35d5a2","task_key":"default--test--10143","task_revision_id":"1","upstream_id":"10143","short_description":"On either side of a pane of window glass, temperatures are 70°F and 0°F. How…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['729 cal/s', '400 cal/s', '850 cal/s', '500 cal/s', '650 cal/s', '1000 cal/s', '300 cal/s', '900 cal/s', '600 cal/s', '475 cal/s']\",\"question\":\"On either side of a pane of window glass, temperatures are 70°F and 0°F. How fast is heat conducted through such a pane of area 2500 cm^2 if the thickness is 2 mm?\",\"src\":\"stemez-Physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}