{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"617903a2-cf40-5df7-8084-197cc9a5ba7e","task_key":"default--test--9947","task_revision_id":"1","upstream_id":"9947","short_description":"In the atomic nucleus, electrical forces tend to","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['cause particles to become neutral', 'increase the mass of the nucleus', 'cause particles to spin', 'generate heat within the nucleus', 'create magnetic fields', 'charge particles', 'push particles apart', 'produce orbital motion', 'hold particles together', 'attract particles to the nucleus']\",\"question\":\"In the atomic nucleus, electrical forces tend to\",\"src\":\"ori_mmlu-conceptual_physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}