{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8617f69b-559d-586e-b27e-2276b7c62843","task_key":"default--test--3108","task_revision_id":"1","upstream_id":"3108","short_description":"During a study session about evolution, one of your fellow students remarks,…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['Genetic traits can be altered by environmental factors after birth.', 'Giraffes with longer necks are more likely to survive, not necessarily the offspring of giraffes that stretch their necks.', 'Traits are inherited randomly, not based on necessity or use.', 'Spontaneous mutations can result in the appearance of new traits.', 'Disuse of an organ may lead to its eventual disappearance.', \\\"The change in the length of the giraffe's neck over generations is a result of genetic drift, not individual effort.\\\", \\\"The giraffe's offspring would only inherit a longer neck if the giraffe's genes were modified.\\\", \\\"Characteristics acquired during an organism's life are generally not passed on through genes.\\\", \\\"The length of a giraffe's neck is determined by its diet and exercise, not its genes.\\\", 'Only favorable adaptations have survival value.']\",\"question\":\"During a study session about evolution, one of your fellow students remarks, \\\"The giraffe stretched its neck while reaching for higher leaves; its offspring inherited longer necks as a result.\\\" Which statement is most likely to be helpful in correcting this student's misconception?\",\"src\":\"ori_mmlu-high_school_biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}