{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"eefed0aa-a845-5879-847c-2347ae764922","task_key":"default--test--3485","task_revision_id":"1","upstream_id":"3485","short_description":"What factors determine the characteristics ofparticular organismsthat exist in…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['Temperature, precipitation, soil chemistry, and the types of decomposers present', 'Only soil structure and chemistry', 'Precipitation, temperature, and the presence of specific predator species', 'Only temperature and precipitation', 'Only the concentration of trace elements', 'The types of rock formations and the pH level of water sources', 'Temperature, soil chemistry, and light exposure', 'Soil structure, light exposure, and the migration patterns of birds', 'Temperature, precipitation, the structure and chemistry of the soil, and the concentration of trace elements', 'The concentration of trace elements and the presence of certain plant species']\",\"question\":\"What factors determine the characteristics ofparticular organismsthat exist in an ecosystem.\",\"src\":\"stemez-Biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}