{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"778f4633-8cfe-5a64-b9b3-614b8f597734","task_key":"default--test--4966","task_revision_id":"1","upstream_id":"4966","short_description":"The Mayan king Yax Pahsaj symbolized his royalty on a stele that depicts:","config":"default","split":"test","body":"{\"category\":\"history\",\"options\":\"['a series of decisive military conquests against a rival king.', 'a depiction of him as a corn god, a symbol of fertility and prosperity.', 'a scene of him hunting and defeating a large jaguar.', 'an eagle, a serpent, and a jaguar, all of which were important royal animals.', 'him receiving a crown from the Mayan god of death and rebirth.', 'his coronation ceremony with all the Mayan gods as witnesses.', 'a lightning bolt descending from heaven and marking him as the leader.', 'a large sun, symbolizing his role as a son of the Sun God.', 'the founder of the dynasty “passing him the torch.”', 'him standing on the peak of a mountain to signify his power.']\",\"question\":\"The Mayan king Yax Pahsaj symbolized his royalty on a stele that depicts:\",\"src\":\"ori_mmlu-prehistory\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}