{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"438fecc8-37a5-5646-8385-d3f24f1cf5d5","task_key":"all--test--438fecc8-37a5-5646-8385-d3f24f1cf5d5","task_revision_id":"1","upstream_id":"","short_description":"A landlord owned an apartment which he leased to a college senior for a…","config":"all","split":"test","body":"{\"choices\":\"['$8,000. 00', '$7,000. 00', '$5,000. 00', '$4,000. 00']\",\"question\":\"A landlord owned an apartment which he leased to a college senior for a three-year term beginning December 1, 2007, for a monthly rental of $1,000 to be payable on the first of each month. After the college senior resided in the apartment for six months, he assigned his interest in the premises to a college junior on June 1,2008. The college junior paid the rental on the apartment for two months and then assigned his interest to a college sophomore. The sophomore took possession on August 1, 2008 and paid the rental for August but did not pay the rental for the next three months of occupancy. Then on December 1, 2008 the college sophomore assigned the premises to a college freshman who took possession but didn't pay any rental to the landlord during the period of his occupancy from December 1,2008 to April 1,2009. On April 5, 2009, the landlord brings an action against the college freshman to recover the outstanding rent that is due on the premises. The freshman is most likely liable for\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}