{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fd236f62-a47a-56c4-ba6f-7b57b344362e","task_key":"default--test--3314","task_revision_id":"1","upstream_id":"3314","short_description":"Fiddler crabs are a dark brown color during the day and change to a pale brown…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"[\\\"The crabs' color change can be influenced by the phase of the moon, reversing the pattern.\\\", 'It is possible to reset the biological clock of the crabs by artificially producing periods of light and darkness or by exposing them to ice water.', 'Crabs can be genetically modified to reverse their day and night coloration patterns.', 'Crabs can change color by exposure to different types of food.', \\\"The crabs' color change pattern can be reversed by playing specific sound frequencies near their habitat.\\\", \\\"The crabs' color change is irreversible and cannot be influenced by external factors.\\\", 'Crabs can change color by adjusting to the temperature of the water.', 'Crabs can change color by changing their habitat.', 'Crabs can change color if they are exposed to certain chemicals in the water.', 'Altering the salinity of the water can cause crabs to reverse their color change pattern.']\",\"question\":\"Fiddler crabs are a dark brown color during the day and change to a pale brown color at night. Is it possible to reverse this pattern and make the crabs turn dark at night and light during the day? How?\",\"src\":\"stemez-Biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}