{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7a770aa8-e136-5c9e-b2a5-d91ff58e5a36","task_key":"default--test--3115","task_revision_id":"1","upstream_id":"3115","short_description":"In the Tularosa Basin of New Mexico are black lava formations surrounded by…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['The light-colored mice evolved from the dark-colored mice over time.', 'The color of the mice is determined by their diet in the lava formations and desert.', 'The two varieties of mice descended from a recent common ancestor.', 'The mouse population was originally one population that diverged into two species because of mutations.', 'Selection favors some phenotypes over others.', 'The two varieties of mice have always existed separately and did not share a common ancestor.', 'Originally the mice were all dark colored. As the lava decomposed into sand, the color of some mice changed because that color was favored in that environment.', 'The mice were originally light-colored, but some darkened over time to blend into the lava formations.']\",\"question\":\"In the Tularosa Basin of New Mexico are black lava formations surrounded by light-colored sandy desert. Pocket mice inhabit both areas. Dark-colored ones inhabit the lava formations, while light-colored mice inhabit the desert. Which of the following statements is correct about this scenario?\",\"src\":\"ori_mmlu-high_school_biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}