{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"38f5b51f-ef37-58ea-a271-c440f10628d6","task_key":"default--test--2526","task_revision_id":"1","upstream_id":"2526","short_description":"Jupiter pilots his newly created perfectionism scale on a high school psychology…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"[\\\"measuring the test's reliability\\\", \\\"Checking the test's validity\\\", 'standardizing the test', \\\"Identifying the test's predictability\\\", \\\"Calculating the test's difficulty level\\\", 'checking for outliers', \\\"Trying to determine the test's sensitivity\\\", 'Analyzing the distribution of scores on the test', \\\"Comparing the test's results with another perfectionism scale\\\", 'looking to see if the mean level of perfectionism has changed']\",\"question\":\"Jupiter pilots his newly created perfectionism scale on a high school psychology class. He returns one month later to administer the same test to the same students, and then he correlates the two sets of results. What is Jupiter probably doing?\",\"src\":\"ori_mmlu-high_school_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}