{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ec736c8e-4546-5d7f-9649-85eca4fad348","task_key":"all--test--ec736c8e-4546-5d7f-9649-85eca4fad348","task_revision_id":"1","upstream_id":"","short_description":"Nancy has the choice to spend one hour studying for an exam, mowing the lawn for…","config":"all","split":"test","body":"{\"choices\":\"['The benefit received from studying is greater than the opportunity cost of $8.', 'The opportunity cost of studying is $14, which is less than the benefit received from studying.', 'Nancy is indifferent to both studying and mowing the lawn.', \\\"Nancy's behavior is irrational, since babysitting was clearly superior to all other options.\\\"]\",\"question\":\"Nancy has the choice to spend one hour studying for an exam, mowing the lawn for one hour at a wage of $6, or babysitting her niece for one hour at a wage of $8. If we know that Nancy has chosen to study for the exam, which of the following is true?\",\"subject\":\"high_school_microeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}