{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5719b709-0dd0-5ab3-bf9c-43beaffae2c1","task_key":"default--test--3094","task_revision_id":"1","upstream_id":"3094","short_description":"Why isn't a biome map of the earth a true representationof thevegetation…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['Biome maps are outdated', 'Biome maps are based solely on temperature, ignoring other ecological factors', 'Biome maps classify vegetation based on animal population distribution rather than plant characteristics', 'Biome maps are too detailed', 'Biome maps use inaccurate color coding that does not reflect the true diversity of vegetation', \\\"Real vegetation types usually grade one into another so it's hard to draw distinct boundaries\\\"]\",\"question\":\"Why isn't a biome map of the earth a true representationof thevegetation formations found on land?\",\"src\":\"stemez-Biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}