{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8d3d4cee-0a9d-521b-9f91-98498963bb80","task_key":"all--test--8d3d4cee-0a9d-521b-9f91-98498963bb80","task_revision_id":"1","upstream_id":"","short_description":"Alice owned Greyacre, a one-acre piece of property, in fee simple. Alice sold…","config":"all","split":"test","body":"{\"choices\":\"['Alice, since her ownership was first in time and she was of record.', 'Barbara, because Charlene had notice of the prior conveyance between Alice and Barbara.', 'Barbara, because she was the first to record her deed.', 'Charlene, because she was the first to record her deed.']\",\"question\":\"Alice owned Greyacre, a one-acre piece of property, in fee simple. Alice sold the property to Barbara who moved into the only residence on the land. Barbara neglected to record the deed, and three years later, Alice conveyed the same property to Charlene who immediately filed the deed she received from Alice. Greyacre's jurisdiction has a race-notice statute and a tract index system. If an action is filed for title to the property, the likely outcome would be for\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}