{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"dde43ca3-deb7-58db-8307-11c075d1e62e","task_key":"default--test--2744","task_revision_id":"1","upstream_id":"2744","short_description":"Discuss the characteristics of verbal materials that influence howeffectively…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['Verbal materials are learned effectively if they are associated with strong emotional content', \\\"Verbal materials are learned based on the learner's prior knowledge of unrelated subjects\\\", \\\"Verbal materials are learned effectively based on the learner's interest\\\", 'Verbal materials are learned based only on their relevance to the subject matter', 'Verbal materials are learned effectively if they are always presented in a rhyming scheme', 'Characteristics of verbal materials that influence how effectively they are learned include list length, item position, meaningfulness, similarity, frequency, concreteness, imagery, and ability to transfer learning.', 'Verbal materials are learned effectively based on the complexity of the language used', 'Verbal materials are learned effectively only when they are presented in a visual format', 'Verbal materials are learned based on the volume and speed at which they are presented', 'Verbal materials are learned based on the time of day they are studied']\",\"question\":\"Discuss the characteristics of verbal materials that influence howeffectively they are learned.\",\"src\":\"stemez-Psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}