{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8ec774f7-d4a7-5173-9107-ddd7209973c6","task_key":"default--test--5669","task_revision_id":"1","upstream_id":"5669","short_description":"In the 1960s, comprehensive schools did not eradicate class inequalities in…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"[\\\"selective schools could still 'cream off' the most able pupils, while the comprehensives put children into ability streams\\\", 'the comprehensive schools did not have the capacity to accommodate all students, leading to overcrowding and reduced quality of education', 'the comprehensive schools were mostly located in affluent neighborhoods, limiting access for working class students', 'teachers in comprehensive schools used the restricted code in the classroom, whereas working class pupils used elaborated codes of speech', 'the segregation between working class and middle class in comprehensive schools was still very apparent', 'they tended to recruit pupils from the inner cities and traditionally working class areas', 'comprehensive schools did not have sufficient funding to provide quality education to all students', 'comprehensive schools did not offer advanced courses, limiting the educational opportunities for gifted students']\",\"question\":\"In the 1960s, comprehensive schools did not eradicate class inequalities in education because:\",\"src\":\"ori_mmlu-sociology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}