{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7b2d6e01-30ea-59ca-a902-eecab1ac1e6c","task_key":"default--test--3147","task_revision_id":"1","upstream_id":"3147","short_description":"Fiddler crabs are a dark brown color during the day and change to a pale brown…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['The crabs can be trained to change color on command using classical conditioning techniques.', \\\"The use of hormonal treatments can induce a reverse in the crabs' color change pattern.\\\", \\\"Painting the crabs' shells can simulate the appearance of a reversed color pattern.\\\", 'The color change can be influenced by the presence of predators, causing a reverse pattern.', 'The crabs can change color based on their mood.', 'It is possible to reset this biological clock by artificially producing periods of light and darkness.', \\\"The pattern is irreversible due to the crabs' hardwired circadian rhythms.\\\", 'The crabs can be forced to change color through genetic manipulation.', \\\"The crabs' color change can be reversed through a diet change.\\\", \\\"The color change can be triggered by altering the temperature of the crabs' environment.\\\"]\",\"question\":\"Fiddler crabs are a dark brown color during the day and change to a pale brown color at night. Is it possible to reverse this pattern and make the crabs turn dark at night and light during the day? How?\",\"src\":\"stemez-Biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}