{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6375a2b3-44ec-5c1c-ac57-c113a3afeb7c","task_key":"all--test--6375a2b3-44ec-5c1c-ac57-c113a3afeb7c","task_revision_id":"1","upstream_id":"","short_description":"What seems to be the case when the folk saying \"opposites attract\" (i.e.,…","config":"all","split":"test","body":"{\"choices\":\"['It does not seem to be true.', 'It seems to be true for heterosexuals but not for homosexuals.', 'It seems to be true for homosexuals but not for heterosexuals.', 'It seems to be true for women but not for men']\",\"question\":\"What seems to be the case when the folk saying \\\"opposites attract\\\" (i.e., dissimilar attitudes tend to cause liking) when it is considered in light of current research?\",\"subject\":\"human_sexuality\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}