{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c37f1b07-47ef-5c0c-9356-1044d87ffd73","task_key":"default--test--3487","task_revision_id":"1","upstream_id":"3487","short_description":"A rooster with gray feathers and a hen with the same phenotype produce 17 gray,…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['The trait for gray is epistatic over the trait for black.', 'The traits for gray and white color demonstrate incomplete dominance.', 'The trait for black is epistatic over the trait for white.', 'The traits for black and white color demonstrate codominance.', 'The trait for white is epistatic over the trait for gray.', 'The traits for black and white color demonstrate incomplete dominance.', 'The traits for black, gray and white colors demonstrate complete dominance.', 'The traits for gray and black color demonstrate codominance.', 'The trait for gray is epistatic over the trait for white.', 'The trait for black is epistatic over the trait for gray.']\",\"question\":\"A rooster with gray feathers and a hen with the same phenotype produce 17 gray, 7 black, and 8 white chicks. What is the simplest explanation for the inheritance of these colors in chickens?\",\"src\":\"ori_mmlu-high_school_biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}