{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"307455da-556f-51b8-9d12-bee483029594","task_key":"all--test--307455da-556f-51b8-9d12-bee483029594","task_revision_id":"1","upstream_id":"","short_description":"To say that \"philosophy\" (like \"love\" or \"art\") is not a closed concept means…","config":"all","split":"test","body":"{\"choices\":\"['there is no one distinguishing feature that identifies an issue as philosophic, only an overlapping of issues roughly associated with one another.', 'the way we come to think about philosophy, love, or art really depends on how we were raised by our families to identify things as resembling one another.', 'the necessary and sufficient condition for something to be considered philosophic is that it answers either of these questions: What does it mean? and How do you know?', 'philosophy is not a closed discipline insofar as it is willing to accept any answer suggested by the \\\"human family\\\" as being true.']\",\"question\":\"To say that \\\"philosophy\\\" (like \\\"love\\\" or \\\"art\\\") is not a closed concept means that we cannot state the necessary and sufficient conditions by which it is defined.  Rather, philosophic issues are identifiable as having \\\"family resemblances\\\" with one another.  In other words:\",\"subject\":\"philosophy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}