{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8da91754-9872-5fae-9bd0-e97ff450020d","task_key":"default--test--3420","task_revision_id":"1","upstream_id":"3420","short_description":"What prevents the stomach from being digested by itsown secretions?","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['The thick muscular wall of the stomach', 'Antibacterial properties of stomach acid', 'Gastric juice', 'High pH levels in the stomach', 'Presence of beneficial bacteria', 'Mucus layer, low permeability of the membranes and continual replacement of cells', 'Hydrochloric acid', 'Stomach lining regeneration every few days', 'Proteolytic enzymes', 'Secretion of digestive hormones']\",\"question\":\"What prevents the stomach from being digested by itsown secretions?\",\"src\":\"stemez-Biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}