{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d16bcf16-a701-53a6-868f-d75dd6a6c69e","task_key":"default--test--2366","task_revision_id":"1","upstream_id":"2366","short_description":"Distinguish between effortful and effortless retrieval, giving examples of each.","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['Effortless retrieval involves a conscious strategy to remember, such as using mnemonic devices, while effortful retrieval happens without conscious strategies.', 'Effortless retrieval is always visual, while effortful retrieval is always verbal', 'Effortful retrieval can happen instantly, like snapping your fingers, while effortless retrieval may take several seconds or minutes, like waking up.', 'Effortless retrieval requires energy and resources, like studying for an exam, while effortful retrieval is low-energy, like daydreaming.', 'Effortless retrieval occurs with frequently used information, like driving a familiar route, while effortful retrieval is for rarely used information, like recalling a historical date.', \\\"Effortless retrieval occurs rapidly with little or no active recall, such as reading a paragraph. Effortful retrieval requires attention and effort, like remembering a familiar person's name.\\\", 'Effortful retrieval involves recalling information without cues, like solving a math problem, while effortless retrieval uses cues, like recognizing a face.', 'Effortless retrieval is for complex tasks requiring analysis, like playing chess, while effortful retrieval is for simple, automatic tasks, like tying shoelaces.', 'Effortless retrieval requires deep concentration, while effortful retrieval is automatic', 'Effortless retrieval is for new information, while effortful retrieval is for familiar information']\",\"question\":\"Distinguish between effortful and effortless retrieval, giving examples of each.\",\"src\":\"stemez-Psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}