{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b128bb9f-bfde-5dbe-a8c4-7808dc320107","task_key":"default--test--1439","task_revision_id":"1","upstream_id":"1439","short_description":"A 15-year-old boy was sledding down a pathway through a wooded area of land…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['not recover, because the boy had been sledding for an extended period without incident.', 'recover, because the woman did not put up any signs forbidding trespassing.', 'recover, because the woman knew or should have known of the frequent trespasses.', 'recover, because the woman did not maintain the safety of her property.', 'not recover, because sledding is inherently dangerous.', 'recover, under the \\\"attractive nuisance\\\" doctrine.', 'recover, because the pathway was indirectly inviting for such activities.', 'not recover, because a 15-year-old boy should have realized the risk.', 'not recover, because the boy was a trespasser.']\",\"question\":\"A 15-year-old boy was sledding down a pathway through a wooded area of land owned by a woman. The boy had frequently used the pathway for sledding in the previous months. The path, made of concrete, led through the woods from a public highway to a pond in the rear of the woman's property. The pathway was used for sledding and the pond for skating by the residents of the neighboring areas, without the consent of the woman. Furthermore, the woman failed to post any signs forbidding trespassing. After the boy had been sledding down the pathway for approximately three hours one morning, he lost control of the sled, and steered it into a tree. The boy suffered serious injuries in the crash. If a suit is brought on the boy's behalf against the woman for the boy's personal injuries, the plaintiff will\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}