{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9b6f1db4-2dc4-5e03-9c5a-8e7f14698d5f","task_key":"default--test--2652","task_revision_id":"1","upstream_id":"2652","short_description":"There are several methods for estimating the test reliability, like the INTERNAL…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['correlating the test results with the age of the test takers', 'correlations among individual test items', 'administering the test to two different groups and then correlating scores', \\\"correlating the test results with the test takers' socio-economic status\\\", 'correlating the test results with another test', 'they are not used for an unstable trait', 'administering the test in multiple languages and then correlating scores', 'administer 2 equivalent forms of a test to same people and correlate', 'measuring the correlation between test results and the time it takes to complete the test', 'administer test twice to same people and then correlate scores']\",\"question\":\"There are several methods for estimating the test reliability, like the INTERNAL CONSISTENCY RELIABILITY for example. From your knowledge about this topic, find the correct definition of it.\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}