{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2c37e859-d2ed-5798-9be7-098306415abc","task_key":"default--test--3515","task_revision_id":"1","upstream_id":"3515","short_description":"Certain nerve gases are known to cause a breakdown of cholinesterase. How would…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['Cause euphoria and hallucinations', 'Improve memory and cognitive functions', 'Increase physical strength and endurance', 'Cause tremors, spasms, or even death', 'Trigger temporary paralysis without pain', 'Enhance sensory perception and reflexes', 'Accelerate healing and tissue regeneration', 'Promote weight loss and increased metabolism', 'Induce relaxation and sleepiness', 'Reduce pain and inflammatory responses']\",\"question\":\"Certain nerve gases are known to cause a breakdown of cholinesterase. How would these gases affect the human body and why?\",\"src\":\"stemez-Biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}