{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2d44ef34-2fb8-5d08-8efb-47df5c5b8ec0","task_key":"all--test--2d44ef34-2fb8-5d08-8efb-47df5c5b8ec0","task_revision_id":"1","upstream_id":"","short_description":"A parent in family A delays for a long time before responding to a crying…","config":"all","split":"test","body":"{\"choices\":\"['for each infant, the length of nighttime crying will depend on the level of arousal', 'family A’s infant will cry for longer periods of time at night', 'family B’s infant will cry for longer periods of time at night', 'neither infant will tend to cry more than the other at any time']\",\"question\":\"A parent in family A delays for a long time before responding to a crying infant. In family B, a crying infant receives immediate attention. At night, a crying infant receives no attention in either family. According to traditional leaning theories, it would be anticipated that\",\"subject\":\"professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}