{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"20e4db16-4fe4-5a47-9431-0a924feb4f59","task_key":"all--test--20e4db16-4fe4-5a47-9431-0a924feb4f59","task_revision_id":"1","upstream_id":"","short_description":"When Berger & Luckmann said that reality is socially constructed, they meant:","config":"all","split":"test","body":"{\"choices\":\"['scientists are guided in their work by social values and interests, so they define and measure phenomena that will support their theories', 'people negotiate shared definitions of their situation and live according to these, often forgetting that these social worlds are not fixed and external', 'sociologists decide what constitutes social reality and measure only that', \\\"terms like 'reality' have no deeper meaning beyond the level of discourse\\\"]\",\"question\":\"When Berger & Luckmann said that reality is socially constructed, they meant:\",\"subject\":\"sociology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}