{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0a4a334a-94e8-5fd6-a4ff-8a5ebefbf7e0","task_key":"default--test--2681","task_revision_id":"1","upstream_id":"2681","short_description":"Discuss the relationship between language and thought. Is language necessary for…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['Language has no relation to thought', 'Thought is entirely dependent on language; without language, humans would be incapable of complex thought.', 'Language is necessary for thought', 'Language is a reflection of thought, and the Whorfian Hypothesis indicates that each language is a direct translation of the same set of thoughts.', 'Language is only a byproduct of thought and does not influence it in any significant way.', \\\"The Whorfian Hypothesis states that language limits and determines thought, making it impossible to think beyond one's language.\\\", 'The Whorfian Hypothesis suggests that language and thought are independent; one does not influence the other.', 'The Whorfian Hypothesis suggests that thinking precedes language', 'Language is an important tool in thinking but not necessary. The Whorfian Hypothesis asserts that the structure of a language influences the thinking process of its speakers.', 'Language and thought are one and the same thing; without language, thought cannot exist.']\",\"question\":\"Discuss the relationship between language and thought. Is language necessary for thought? What is the Whorfian Hypothesis?\",\"src\":\"stemez-Psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}