{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4e3621e7-e776-563a-99d3-32eabc9f460f","task_key":"default--test--2522","task_revision_id":"1","upstream_id":"2522","short_description":"The Self-Concept is the individual’s belief about their own attributes. What can…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['to experience an emotion, a person must first experience cognitive interpretation, then physiological arousal', 'not rewarding people for an enjoyable activity can undermine their interest', 'not rewarding people for an unenjoyable activity can enhance their interest', 'when internal cues are clear, people infer by observing their behavior and the situational context', 'rewarding people for an unenjoyable activity can enhance their interest', 'to experience an emotion, a person must first experience physiological arousal, then a cognitive interpretation of that arousal', 'when internal cues are vague, people infer by observing their behavior and the situational context', 'rewarding people for an enjoyable activity can undermine their interest', 'when external cues are vague, people infer by observing their behavior and the situational context', 'rewarding people excessively can enhance their interest']\",\"question\":\"The Self-Concept is the individual’s belief about their own attributes. What can happen according to the OVERJUSTIFICATION HYPOTHESIS?\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}