{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1aa44e2c-8b2b-5e2b-8eec-319c054e0e55","task_key":"default--test--3295","task_revision_id":"1","upstream_id":"3295","short_description":"Why must mosses and liverworts (phylumBryophyta) always live in close…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['They release spores that can only germinate in water', 'They have no roots, so they must be submerged in water to remain upright', 'They require high humidity environments but do not need direct contact with water', 'Their leaves are adapted to absorb water directly from the atmosphere', 'They lack efficient water absorption and transport systems, and their reproduction requires water.', 'They are aquatic plants', 'They use water as a habitat to avoid predators', 'They need water to photosynthesize', 'They cannot survive in dry environments']\",\"question\":\"Why must mosses and liverworts (phylumBryophyta) always live in close association with the water?\",\"src\":\"stemez-Biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}