{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7754a5fc-6aad-577c-aa5d-8d7468374e06","task_key":"default--test--2029","task_revision_id":"1","upstream_id":"2029","short_description":"An animal is not likely to associate a conditioned stimulus (CS) with an…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['delay between the CS and the UCS is too long', 'interval schedule between the two stimuli is variable', 'UCS and CS are presented simultaneously', 'interstimulus ratio is variable', 'UCS is significantly stronger than the CS', 'fixed-rational schedule between the two stimuli is altered', 'CS and UCS are of equal intensity']\",\"question\":\"An animal is not likely to associate a conditioned stimulus (CS) with an unconditioned stimulus (UCS) if the\",\"src\":\"ori_mmlu-high_school_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}