{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6f60f762-5827-5880-97eb-d2c051b6d7b3","task_key":"default--test--2606","task_revision_id":"1","upstream_id":"2606","short_description":"Discuss the variables which determine whether transfer will be positive or…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['The duration of the tasks affects the type of transfer', 'Transfer is dependent on the time of day the tasks are performed', 'Positive or negative transfer is determined solely by the age of the individual', 'The physical environment where the tasks are performed, such as indoors or outdoors, determines the nature of transfer', 'The number of participants involved in the tasks affects the transfer outcome', 'The degree of similarity of the stimuli and responses in two tasks determines whether transfer will be positive or negative.', 'The type of transfer is determined by the difficulty level of the tasks', 'The presence of background noise during task execution dictates the transfer type', 'The complexity of the tasks determines positive or negative transfer', 'The emotional state of the individual during task performance influences the direction of transfer']\",\"question\":\"Discuss the variables which determine whether transfer will be positive or negative.\",\"src\":\"stemez-Psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}