{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"93c712af-2552-5393-912c-439b2b8fe0a0","task_key":"default--test--2133","task_revision_id":"1","upstream_id":"2133","short_description":"The STANDARD SCORES express raw score distance from mean by SD and considered…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['M=0, SD=1, often used in research studies', 'M=100, SD=15 or 16', 'They express raw scores in terms of percentiles', 'M=500, SD=100, often used in college entrance exams', 'Shows how many standard deviations a score is above or below the mean: a score of +1.0 means it’s 1 SD above the mean', 'M=100, SD=10, commonly used in IQ tests', 'M=100, SD=24, typically used in personality assessments', 'M=50, SD=10- it is used on many psychological tests, MMPI', 'They express raw scores in terms of ranks', 'standard nine: M=5, SD=approx 2']\",\"question\":\"The STANDARD SCORES express raw score distance from mean by SD and considered the most satisfactory type of NR score. Which statement out of the following ones is about T-scores?\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}