{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3a3024f9-f3b3-59bb-abf7-0d87082d03c8","task_key":"default--test--9303","task_revision_id":"1","upstream_id":"9303","short_description":"In which of the following situations involving a source of sound and a detector…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['Both the source and detector are rotating around a common axis.', 'The detector travels toward the stationary source.', 'Both the source and detector travel in the same direction.', 'The source travels away from the stationary detector.', 'Both the source and detector travel in opposite directions, with the source and detector moving away from each other.', 'The source travels toward the stationary detector.', 'The source and detector travel in opposite directions, with the source moving towards the detector.', 'The detector travels away from the stationary source.']\",\"question\":\"In which of the following situations involving a source of sound and a detector of the sound is it possible that there is NO perceived Doppler shift?\",\"src\":\"ori_mmlu-high_school_physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}