{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a9b5a8db-08a5-5661-ae43-98b455593e2e","task_key":"default--test--2554","task_revision_id":"1","upstream_id":"2554","short_description":"How do motor or enactive codes differ from visual or verbal codes. Illustrate…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['Enactive codes store motor activity and can be utilized without active, conscious attention, unlike visual and verbal codes.', 'Visual codes are less detailed and less precise than enactive codes', 'Enactive and verbal codes are stored in the same way in the brain', 'Enactive codes require active, conscious attention', 'Enactive codes and visual codes are the same', 'Enactive codes are a type of verbal code used for complex problem-solving', 'Enactive codes are visual representations', 'Visual codes can be recalled without having ever been performed', 'Enactive codes are always accompanied by verbal explanations', 'Verbal codes are not based on language but on motor skills']\",\"question\":\"How do motor or enactive codes differ from visual or verbal codes. Illustrate with examples.\",\"src\":\"stemez-Psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}