{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1521ab55-10c3-5b19-a5bc-a2d060855823","task_key":"all--test--1521ab55-10c3-5b19-a5bc-a2d060855823","task_revision_id":"1","upstream_id":"","short_description":"A pet breeder is in the business of breeding calves at his cattle ranch where he…","config":"all","split":"test","body":"{\"choices\":\"['Yes, because he had a limited privilege to enter the land to prevent harm to his chattel.', 'Yes, because the tender pet doctrine allows temporary entry to retrieve baby animals.', 'No, because the neighbor had a right to keep any living chattels that crossed onto his land.', 'No, because his status as a breeder made him unqualified for a limited license.']\",\"question\":\"A pet breeder is in the business of breeding calves at his cattle ranch where he has a stable of prolific cows who are very fertile. The newborn calves need constant attention and care. One day one of the employees inadvertently leaves the fence door open and a newly-born calf breaks free and goes to his neighbor's land. The breeder went to the neighbor's land to retrieve the calf for its safety and to make sure it was unharmed. However, he was arrested on a trespass charge after entering the land. The breeder appealed. Will the court dismiss the charge?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}