{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ffa8b621-3024-54bc-92c8-fe29a60522c8","task_key":"default--test--2319","task_revision_id":"1","upstream_id":"2319","short_description":"What is validity?","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['Validity is the extent to which a test can be administered in different languages.', 'Validity is whether the test measures what the examiner wants it to measure and how well it measures it.', 'Validity is based solely on empirical evidence.', 'Validity is the degree to which a test predicts future performance.', 'Validity is the reliability of a test.', 'Validity is the level of difficulty of a test.', 'Validity is the length of time a test remains applicable.', 'Validity is the ease with which test results can be compared across different populations.', 'Validity is the consistency of test results over time.', 'Validity is the subjective interpretation of a test.']\",\"question\":\"What is validity?\",\"src\":\"stemez-Psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}