{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a19e7b54-6389-5b94-bb60-b4d91c316fe9","task_key":"default--test--5771","task_revision_id":"1","upstream_id":"5771","short_description":"In a holistic evaluation of student essays, evaluations are made on the basis of…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"[\\\"overall quality of each student's essay in relation to the topic\\\", \\\"number of words used in each student's essay\\\", \\\"number of paragraphs in each student's essay\\\", 'number and variety of errors made by each student', \\\"use of metaphors and similes in each student's essay\\\", 'the grade level each student is in', 'ability of each student to communicate in a variety of discourse modes', \\\"handwriting quality of each student's essay\\\", 'average sentence length and the complexity demonstrated in each essay', 'the amount of time each student took to write the essay']\",\"question\":\"In a holistic evaluation of student essays, evaluations are made on the basis of the\",\"src\":\"ori_mmlu-miscellaneous\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}