{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d733b794-447a-5951-a9d1-a6dc9bb65e56","task_key":"default--test--2297","task_revision_id":"1","upstream_id":"2297","short_description":"In the Harlow study of emotional attachment, infant monkeys were placed in a…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['the type of stimuli introduced to the monkeys', 'with which \\\"mother\\\" the bottle of milk is placed', 'the gender of the infant monkeys', 'the amount of milk in the bottle', 'the type of material used to create the \\\"mothers\\\"', 'the \\\"wire\\\" mother versus the \\\"cloth\\\" mother', 'the age of the infant monkeys', 'the time of day the milk bottle was moved', 'the preference of the infants for the source of milk', 'the preference of the infants for the \\\"wire\\\" mother']\",\"question\":\"In the Harlow study of emotional attachment, infant monkeys were placed in a cage and given both a \\\"wire\\\" mother and a \\\"cloth\\\" mother. Researchers then moved a bottle of milk from one mother to the other while introducing various stimuli to see if the monkeys would form an attachment to either of the \\\"mothers.\\\" In this experiment, the independent variable is\",\"src\":\"ori_mmlu-high_school_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}