{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"88fe6493-3272-55c0-a0ec-66baf5ec2e29","task_key":"default--test--3101","task_revision_id":"1","upstream_id":"3101","short_description":"What factors determine the characteristics ofparticular organismsthat exist in…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['Temperature, humidity, and human activities', 'Only soil structure and chemistry', 'Only sunlight and humidity', 'Sunlight, soil pH, and predator-prey relationships', 'Only temperature and precipitation', 'Precipitation, geothermal activity, and biodiversity levels', 'Soil structure, water salinity, and atmospheric pressure', 'Temperature, precipitation, soil structure and chemistry, sunlight, humidity, and concentration of trace elements', 'Sunlight, temperature, and migration patterns of nearby species', 'Precipitation, altitude, and mineral composition of rocks']\",\"question\":\"What factors determine the characteristics ofparticular organismsthat exist in an ecosystem.\",\"src\":\"stemez-Biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}