{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b8d5f9f9-e430-5585-948b-c8c93770cf91","task_key":"default--test--2362","task_revision_id":"1","upstream_id":"2362","short_description":"List and describe the scientific characteristics of psychology.","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['Psychology is a natural science that focuses on the study of inanimate objects', 'Psychology is based on intuition and opinion', 'Psychology is not empirical', 'Psychology is empirical, systematic, and employs measurement techniques.', 'Psychology uses random observation without systematic methods or replication', 'Psychology studies only the collective behavior of groups, ignoring individual differences', 'Psychology is solely concerned with animal behavior and does not apply to humans', 'Psychology relies solely on theoretical approaches without practical experimentation', 'Psychology is a pseudoscience that cannot make predictions or be falsified', 'Psychology does not employ measurement techniques']\",\"question\":\"List and describe the scientific characteristics of psychology.\",\"src\":\"stemez-Psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}