{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"89664337-cefa-5442-a322-20f9dd5c5c1b","task_key":"default--test--9731","task_revision_id":"1","upstream_id":"9731","short_description":"When a ball at rest hangs by a single vertical string tension in the string is…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['may be equal to mg depending on the speed of the ball', 'is mg', 'is less than mg when the ball is at the highest point in the circle', 'is greater than mg always', 'is equal to the square of the speed of the ball', 'remains unchanged regardless of the movement of the ball', 'is inversely proportional to the speed of the ball', 'is less than mg always', 'is directly proportional to the speed of the ball', 'may be greater or less than mg depending on the speed of the ball']\",\"question\":\"When a ball at rest hangs by a single vertical string tension in the string is mg. If the ball is made to move in amhorizontal circle so that the string describes a cone string tension\",\"src\":\"ori_mmlu-conceptual_physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}