{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f4065817-05c4-5738-b0c1-fc3f1bbe3475","task_key":"default--test--9464","task_revision_id":"1","upstream_id":"9464","short_description":"In one type of mass spectrometer the charged particles pass through a velocity…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['1.15 and 1.07 (close to correct but slightly off)', '0.9 and 1.0 (near to correct but incorrect order)', '1.4 and 1.5 (exaggerated difference)', '1.2 and 1.3', '0.7 and 0.8 (incorrect proportions)', '0.75 and 0.85 (similar but incorrect)', '0.857 and 0.926', '0.926 and 0.857', '1.0 and 1.0 (suggesting no difference in radii)', '0.5 and 0.6']\",\"question\":\"In one type of mass spectrometer the charged particles pass through a velocity selector before entering the magnetic field. In another the particles pass through a strong electric field before entering the magnetic field. Compare the ratio of the radii of singly charged lithium ions of masses 6amuand 7amuin the two cases.\",\"src\":\"stemez-Physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}