{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"87b4a3c4-7fb7-577b-bc9e-67e7e08812bc","task_key":"all--test--87b4a3c4-7fb7-577b-bc9e-67e7e08812bc","task_revision_id":"1","upstream_id":"","short_description":"Farmer Fred was the owner of Grapeview Farm, the largest farm in the County of…","config":"all","split":"test","body":"{\"choices\":\"['Insufficient, because it is too vague.', 'Sufficient, because it satisfies the title of seisin.', 'Sufficient, because the property can be identified.', 'Insufficient, because the gifts to the children lacked consideration.']\",\"question\":\"Farmer Fred was the owner of Grapeview Farm, the largest farm in the County of Grapeview. Fred was advised by his gift and estate attorney to begin making annual fractionalized gifts of the property to his children. Fred executed a conveyance instrument, stating the gifts as \\\"one-thirtieth of my ownership interest in Grapeview County, the eastern one tenth of the western corner of the property is hereby given to all my children per capita.\\\" This description of the land to be conveyed is\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}