{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1f442588-9a3c-5c98-8753-93113fb0ce00","task_key":"default--test--1079","task_revision_id":"1","upstream_id":"1079","short_description":"A 12-year-old girl and her friends were playing catch with a baseball in the…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['recover for personal injury.', 'not recover as she was at fault for not paying attention.', 'recover for battery only.', 'recover for assault and battery.', 'recover for both negligence and assault.', 'recover for intentional infliction of emotional distress.', 'recover for negligence only.', 'recover for assault only.', 'not recover.', 'recover for both negligence and battery.']\",\"question\":\"A 12-year-old girl and her friends were playing catch with a baseball in the middle of a field of grass in the park. Near the edge of the field, a woman was sitting in a beach chair and reading a book. The girl threw the ball too far, and it went over her friends' heads and flew toward the woman. Although the woman did not see the ball coming, it hit the straw hat that the woman was wearing, knocking it from her head. Although the woman was not touched by the ball, she was startled by the ball hitting her hat, and she fell from her chair and broke her arm. If the woman initiates a suit against the girl to recover damages for her broken arm, the woman will\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}