{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7b2e8907-c119-5ae1-9c80-5b3630392da9","task_key":"default--test--9563","task_revision_id":"1","upstream_id":"9563","short_description":"A bar with a cross-sectional area of 6 square centimeters and a permeability of…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['6000 lines of flux', '45,000 lines of flux', '40,000 lines of flux', '5000 lines of flux', '25,000 lines of flux', '20,000 lines of flux', '35,000 lines of flux', '12,000 lines of flux', '15,000 lines of flux', '30,000 lines of flux']\",\"question\":\"A bar with a cross-sectional area of 6 square centimeters and a permeability of 1000 units is placed in a magnetic field of intensity 5 oersteds. What is the total number of lines of magnetic flux in the iron?\",\"src\":\"stemez-Physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}