{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fd89b04e-412a-5ebf-9bd7-2bd1df2b989e","task_key":"all--test--fd89b04e-412a-5ebf-9bd7-2bd1df2b989e","task_revision_id":"1","upstream_id":"","short_description":"A hiker was enjoying a journey through a large forest when he was caught in an…","config":"all","split":"test","body":"{\"choices\":\"['The hiker does not owe anything because the importance of his survival outweighs the property interests of the cabin owner.', 'The hiker must pay for the cost of the food and the rental value of the cabin for one night.', 'The hiker must pay for the use of the cabin, but nothing else.', 'The hiker must pay for the cost of the food, but nothing else.']\",\"question\":\"A hiker was enjoying a journey through a large forest when he was caught in an unexpected blizzard. As it grew dark and the snow continued, he realized he was completely lost. He felt sure that he would freeze to death during the night. When all hope seemed lost, he stumbled across a small cabin. He knocked on the door, but no one answered. He tried to open the door, realized it was not locked, and went inside. In the cabin, he found some food to eat and some blankets that kept him warm through the night. By the next morning, the snow had stopped, and he was able to make his way out of the forest. If the owner of the cabin sues the hiker, seeking to recover damages for the cost of the food and for the use of his cabin, what is the most likely outcome of the case?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}