{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3666b7f3-6af7-5af2-b00f-36ba7f397284","task_key":"all--test--3666b7f3-6af7-5af2-b00f-36ba7f397284","task_revision_id":"1","upstream_id":"","short_description":"The Secretary of Homeland Security administers and enforces the immigration…","config":"all","split":"test","body":"{\"choices\":\"['Yes because the Sheriff is stressing a speculative possibility that his office may incur funding shortages and more enforcement problems at some undefined future time.', 'Yes, because the Sheriff, as a part of the executive branch himself, is not legally qualified to bring a claim against the executive.', 'No, because anyone has the right to object to the executive branch when it tries to legislate its own opinion into the existing law.', \\\"No, because the Sheriff's county is near the border and it will in fact suffer an ingestion of more immigrants from the policy.\\\"]\",\"question\":\"The Secretary of Homeland Security administers and enforces the immigration laws. 8 U.S.C. § 1103(a)(1). The Secretary has the discretion to implement \\\"deferred action,\\\" which temporarily postpones the exportation of certain groups of undocumented immigrants. With 11.3 million people now here illegally, the Secretary implemented a program called the Deferred Action for Childhood Arrivals, or DACA. The program deferred removal for two years of individuals who came to the United States as children, met certain eligibility criteria, and cleared a background check. A Sheriff in a county near the Mexican border, sued the Secretary, claiming that the policy would increase crime and drain resources in his county. The U.S. District Court dismissed the Sheriff's claim for lack of standing. On appeal, will the U.S. Circuit Court of Appeals likely affirm the dismissal?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}