{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"053a02e1-b336-5134-a7f7-0614bb6c54ff","task_key":"default--test--3250","task_revision_id":"1","upstream_id":"3250","short_description":"Arthropods have an exoskeleton. How is movement accomplished?","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['They use their wings to move', 'They move by changing the shape of their exoskeleton', 'They use cilia on the surface of their exoskeleton to move', 'They move due to their muscular system', 'They have a soft, pliable inner layer that bends to facilitate movement', 'Movement is achieved through hydraulic pressure changes within each segment', 'They release a special fluid that lubricates the joints of the exoskeleton for movement', 'The exoskeleton contracts and expands allowing for movement', 'Movement is achieved by the exoskeleton splitting and reforming in new positions', 'The body is segmented and the segments are joined by a thin layer of flexible chitin']\",\"question\":\"Arthropods have an exoskeleton. How is movement accomplished?\",\"src\":\"stemez-Biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}