{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8df56857-e35b-5028-bd25-8a17038e287b","task_key":"default--test--3406","task_revision_id":"1","upstream_id":"3406","short_description":"What are the distinguishing features of the several orders of insects?","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['Insects are categorized by the number of legs and the shape of their antennae', 'Insects are divided based on their color and size', 'Insects are differentiated by the type of soil they inhabit and their resistance to environmental changes', 'Insects are divided into different orders based primarily upon wing structure, type of mouthparts, and the type of metamorphosis they undergo. The class Insecta is divided into two subclasses, Apterygota and Pterygota.', 'Orders of insects are determined by the presence or absence of a coelom and the type of respiratory system', 'Insects are grouped based on their method of locomotion and feeding times', 'Insects are divided based on their habitat and diet', 'Insects are classified according to their ability to produce sound and their mating rituals', 'Insects are divided based on their lifespan and reproduction method', 'Insect classification hinges on their social behavior and the complexity of their nests or colonies']\",\"question\":\"What are the distinguishing features of the several orders of insects?\",\"src\":\"stemez-Biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}