{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f37e572e-0ec4-59f0-93c3-d90b7b6f15c6","task_key":"default--test--9814","task_revision_id":"1","upstream_id":"9814","short_description":"A whiffle ball is tossed straight up, reaches a highest point, and falls back…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['I & III only', 'None of the above', 'I, II & III', 'I & II only', 'II only', 'III only', 'II & III only', 'I & II & III', 'I only']\",\"question\":\"A whiffle ball is tossed straight up, reaches a highest point, and falls back down. Air resistance is not negligible. Which of the following statements are true?\\nI. The ball’s speed is zero at the highest point.\\nII. The ball’s acceleration is zero at the highest point.\\nIII. The ball takes a longer time to travel up to the highest point than to fall back down.\",\"src\":\"ori_mmlu-high_school_physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}