{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"59ab746a-5dc7-5156-b38f-1aa04fe92289","task_key":"default--test--9254","task_revision_id":"1","upstream_id":"9254","short_description":"Which of the following statements about sidereal and solar days is not true?","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['A sidereal day is the time it takes for a star to return to the same place in our sky.', 'A sidereal day represents exactly 360° of rotation for the earth.', 'A solar day is 4 minutes longer than a sidereal day.', 'A solar day represents more than 360° of rotation for the earth.', 'A sidereal day is 4 minutes shorter than a solar day.', 'The time it takes for the Moon to make one circuit of our sky is one solar day.', 'The time it takes for the Sun to make one circuit of our sky is one solar day.']\",\"question\":\"Which of the following statements about sidereal and solar days is not true?\",\"src\":\"ori_mmlu-astronomy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}