{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f5a1cb35-f1c0-5d05-9efb-b88805dc5bfb","task_key":"default--test--5749","task_revision_id":"1","upstream_id":"5749","short_description":"Keith is really anxious about a big presentation he has to give at work…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['relaxation inhibits the parasympathetic system', 'anxiety inhibits the sympathetic system', 'anxiety increases blood flow to the lower spinal cord inhibiting vasocongestion', 'relaxation is necessary for vasocongestion', 'anxiety decreases blood flow to the upper spinal cord inhibiting vasocongestion', 'anxiety causes an increase in adrenaline which promotes arousal', 'anxiety causes retrograde ejaculation which inhibits arousal', \\\"Keith's problem is not related to his anxiety but due to physical issues\\\", 'relaxation causes retrograde ejaculation which inhibits arousal', 'anxiety causes an increase in testosterone which inhibits arousal']\",\"question\":\"Keith is really anxious about a big presentation he has to give at work tomorrow. As a result tonight he is having trouble getting an erection. Which of the following BEST explains why Keith might be having this problem?\",\"src\":\"ori_mmlu-human_sexuality\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}