{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0070b189-7696-5a28-a3a1-93e19bacbf80","task_key":"default--test--10995","task_revision_id":"1","upstream_id":"10995","short_description":"Shapiro claims that surveys have been confirmed by longitudinal studies that…","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['the use of illegal drugs is usually a temporary phase in adolescence.', 'most individuals experiment with illegal drugs but do not continue use.', 'excessive use of legal drugs is the norm.', \\\"an individual's illegal drug use inevitably diminishes over time.\\\", 'those who begin using illegal drugs typically stop before addiction sets in.', 'longitudinal studies have shown no consistent patterns in illegal drug use.', 'illegal drug users typically transition to legal substances over time.', 'moderate and/or controlled use of illegal drugs is the norm.', 'those who begin using illegal drugs typically do not stop until they become addicted.', 'excessive use of illegal drugs is the norm.']\",\"question\":\" Shapiro claims that surveys have been confirmed by longitudinal studies that indicate that\",\"src\":\"ori_mmlu-moral_disputes\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}