{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c2767ec5-833e-5b87-830b-49e7c50cb3e6","task_key":"default--test--2897","task_revision_id":"1","upstream_id":"2897","short_description":"About 0.5 percent of the American population have epilepsy, the falling…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['Epilepsy is the result of a chromosomal aneuploidy, similar to Down syndrome.', 'Epilepsy is always inherited in an X-linked recessive pattern.', 'Epilepsy is a dominant trait.', 'Epilepsy is caused by an autosomal dominant trait.', 'Epilepsy is due to an autosomal recessive trait.', 'Epilepsy is caused by an autosomal dominant gene with complete penetrance.', 'Epilepsy is a mitochondrial inheritance disorder.', 'Epilepsy is exclusively caused by a single gene mutation.', 'Epilepsy is not a genetic disorder.', 'Epilepsy is a polygenic disorder with no environmental factors involved.']\",\"question\":\"About 0.5 percent of the American population have epilepsy, the falling sickness. Discuses the genetics of this disease.\",\"src\":\"stemez-Genetics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}