{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2ec34ad1-8f58-5577-aac9-a89c2c907f72","task_key":"default--test--2993","task_revision_id":"1","upstream_id":"2993","short_description":"A piece of potato is dropped into a beaker of pure water. Which of the following…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['Water moves from the surrounding water into the potato.', 'Solutes in the water move into the potato.', 'No movement of water or solutes occurs.', 'Potato cells plasmolyze.', 'Solutes in the potato move into the water.', 'Water and solutes move equally in both directions.', 'The potato dissolves in the water.', 'Water moves from the potato into the surrounding water.', 'The potato absorbs all of the water.', 'The potato cells burst due to water intake.']\",\"question\":\"A piece of potato is dropped into a beaker of pure water. Which of the following describes the activity after the potato is immersed into the water?\",\"src\":\"ori_mmlu-high_school_biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}