{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6d949079-1a45-502e-9258-4cc04dfe1ce7","task_key":"all--test--6d949079-1a45-502e-9258-4cc04dfe1ce7","task_revision_id":"1","upstream_id":"","short_description":"Identify the conclusion of the following argument. That the world is my idea is…","config":"all","split":"test","body":"{\"choices\":\"['The world is an idea.', 'That the world is my idea is a truth valid for every living creature, though only man can contemplate it.', 'In doing so, he attains philosophical wisdom.', 'No truth is more absolutely certain than that all that exists for knowledge and therefore this world is only object in relation to subject, perception of a perceiver.']\",\"question\":\"Identify the conclusion of the following argument. That the world is my idea is a truth valid for every living creature, though only man can contemplate it. In doing so, he attains philosophical wisdom. No truth is more absolutely certain than that all that exists for knowledge and therefore this world is only object in relation to subject, perception of a perceiver. The world is an idea. (Arthur Schopenhauer, The World as Will and Idea)\",\"subject\":\"formal_logic\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}