{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"953c50d4-7bd9-5099-bf59-70b397ba8d83","task_key":"default--test--2088","task_revision_id":"1","upstream_id":"2088","short_description":"A person who is low in self-monitoring (Snyder, 1987) will rely on which of the…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['his/her life scripts.', 'past experiences in similar situations.', 'the advice of friends or family.', 'random decision making.', 'societal norms and expectations.', 'internal working models that were acquired during infancy and early childhood.', 'popular trends or fads.', 'the emotional reactions and behaviors of the people he/she is interacting with.', 'the expectations of authority figures.', 'his/her own feelings, attitudes, and beliefs.']\",\"question\":\"A person who is low in self-monitoring (Snyder, 1987) will rely on which of the following when deciding how to act in a particular social situation?\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}