{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1b1d4351-24f5-5b4e-b0df-9c8b5b71454c","task_key":"default--test--2550","task_revision_id":"1","upstream_id":"2550","short_description":"Describe the structural theory of language acquisition. Define generative…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['The structural theory of language acquisition maintains that language acquisition is an innate capacity and humans are governed by certain innate principles. Generative grammar is the set of rules that determine how the language is structured.', 'Generative grammar is a dictionary of all the words in a language', 'Language acquisition depends entirely on social interaction without any innate mechanisms', 'Generative grammar is the restrictive rules of a language', 'Language acquisition is a learned behavior', 'The structural theory posits that language is learned through conditioning and reinforcement, and generative grammar is the catalog of phrases used in daily communication', 'Language acquisition is solely a product of direct instruction and imitation', 'Language acquisition is not correlated with motor development', 'Generative grammar refers to the universal ability to create an infinite number of sentences using a finite set of elements', 'The structural theory suggests that language acquisition is primarily influenced by environmental factors']\",\"question\":\"Describe the structural theory of language acquisition. Define generative grammar.\",\"src\":\"stemez-Psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}