{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"01d99447-492a-59cf-8c3c-412e4da01aa1","task_key":"default--test--10264","task_revision_id":"1","upstream_id":"10264","short_description":"The Relativistic Heavy Ion Collider (RHIC) at the Brookhaven National Laboratory…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['0.75996', '0.65996', '0.55996', '0.69996', '0.85996', '0.50996', '0.79996', '0.89996', '0.94996', '0.99996']\",\"question\":\"The Relativistic Heavy Ion Collider (RHIC) at the Brookhaven National Laboratory collides gold ions onto other gold ions head on. The energy of the gold ions is 100 GeV per nucleon. What is the speed of the gold ions as a fraction of the speed of light?\",\"src\":\"theoremQA-Physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}