{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b5455c93-1ebb-5b0f-b6b4-0609228277a3","task_key":"default--test--9125","task_revision_id":"1","upstream_id":"9125","short_description":"When you brush your hair and scrape electrons from your hair, the charge of your…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['The charge depends on the type of brush used', 'The charge varies depending on the type of hair', 'Neutral', 'Neither of these', 'negative', 'positive', 'The charge becomes zero', 'The charge is alternately positive and negative', 'Both of these', \\\"The charge can't be determined by brushing\\\"]\",\"question\":\"When you brush your hair and scrape electrons from your hair, the charge of your hair becomes\",\"src\":\"ori_mmlu-conceptual_physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}