{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0869a376-5c0a-5938-9bf9-b3059a81ddc8","task_key":"default--test--3453","task_revision_id":"1","upstream_id":"3453","short_description":"How and where is seminal fluid produced in man?","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['Seminal fluid is produced by the thyroid gland.', 'Seminal fluid is produced by the bladder.', 'Seminal fluid is produced by the pituitary gland.', \\\"Seminal fluid is produced by the paired seminal vesicles, the prostate gland, and Cowper's glands in man.\\\", 'Seminal fluid is produced by the pancreas.', 'Seminal fluid is produced by the liver.', 'Seminal fluid is produced by the epididymis.', 'Seminal fluid is produced by the urethra.', 'Seminal fluid is produced by the adrenal glands.', 'Seminal fluid is produced by the testes.']\",\"question\":\"How and where is seminal fluid produced in man?\",\"src\":\"stemez-Biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}