{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9d6bbbf1-57e5-5802-ba4d-7aa6c5c4a88c","task_key":"default--test--2102","task_revision_id":"1","upstream_id":"2102","short_description":"Lizette and her family watch the sunset over the ocean. While walking home in…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"[\\\"Lizette's eyes are adjusting to the darkness, hence she cannot distinguish the colors of objects.\\\", 'Cones, which are specialized for color vision, require more light to be activated, whereas rods, which produce images in black, white, and gray, allow for vision in dim light.', 'The retina does not respond to changes in light intensity, therefore colors are not visible in dim light.', 'The receptive fields in the retina respond to the loss of light through light adaptation, the process of rapidly adjusting to a diminution of light.', 'Cones, which are specialized for black and white vision, require a small amount of light to be activated, whereas rods, which produce images in color, require greater amounts of light for vision.', 'Rods and cones both require a high amount of light to be activated, hence color vision is not possible in dim light.', 'The cones, which are responsible for color vision, are deactivated in dim light.', 'The color vision is not possible in the dark because the photoreceptor cells in the retina require light to detect colors.', 'The colors are not visible in dim light because the rods, which detect colors, are deactivated.', 'Rods, which are specialized for color vision, require more light to be activated, whereas cones, which produce images in black, white, and gray, allow for vision in dim light.']\",\"question\":\"Lizette and her family watch the sunset over the ocean. While walking home in the increasing darkness, Lizette notices that she can no longer distinguish the colors of objects. Which of the following best explains why Lizette cannot see color in dim light?\",\"src\":\"ori_mmlu-high_school_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}