{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cccd98bc-f63b-52e1-a420-0f4eaf784924","task_key":"default--test--3186","task_revision_id":"1","upstream_id":"3186","short_description":"How does the wing shape of a bird correlate with its distinctive flight?","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['Birds with more pointed wings are less agile in the air.', \\\"Wing shape has no impact on a bird's flight.\\\", 'The shape of a wing is correlated with both the power and type of flight.', 'Birds with smaller wings fly faster.', 'Only the size of the wings matter for flight.', 'Birds with longer wings cannot fly as far as those with shorter wings.', 'The color of the wings is the primary factor determining flight characteristics.', 'All bird wings are similar in shape, leading to similar flight patterns.', 'Wing texture is more important than wing shape for flight efficiency.', 'The wing shape only affects takeoff and landing, not sustained flight.']\",\"question\":\"How does the wing shape of a bird correlate with its distinctive flight?\",\"src\":\"stemez-Biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}