{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b7ea62d8-e6f4-5ba0-ab1b-7ca7c454fc7e","task_key":"all--test--b7ea62d8-e6f4-5ba0-ab1b-7ca7c454fc7e","task_revision_id":"1","upstream_id":"","short_description":"A consumer watchdog group presented petitions to the state legislature bearing…","config":"all","split":"test","body":"{\"choices\":\"['no relief, because easements for utility lines are presumed to be beneficial to the servient estate.', 'no relief, because the legislation is merely a regulation of the use of property and not a taking.', 'no relief, because the corporation is not a government entity.', 'damages for the value of property used by the corporation.']\",\"question\":\"A consumer watchdog group presented petitions to the state legislature bearing the signatures of over 10,000 state residents complaining about the recent increases in the cost of cable television. A corporation who provides cable television services successfully persuaded the state legislature to grant it an exclusive right to install cable television lines in all multiple family dwellings in the state, in exchange for the corporation's promise to freeze cable television rates for the next four years. An owner of several large multifamily apartment buildings in the state brought an action in federal district court challenging the constitutionality of the state legislation. The suit claimed that the space used by the corporation when it subsequently installed cable television lines in one of his apartment buildings amounted to a taking without compensation. In this action, the owner will be awarded\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}