{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d8117951-3d9a-5e96-a9c1-94cd7bb86005","task_key":"all--test--d8117951-3d9a-5e96-a9c1-94cd7bb86005","task_revision_id":"1","upstream_id":"","short_description":"When cross-validation of a set of predictors results in great shrinkage of an…","config":"all","split":"test","body":"{\"choices\":\"['the selection of weights for the predictors capitalized on chance', 'none of the predictor variables was sufficiency reliable', 'the number of independent variables used in the first study was too small', 'the criterion variable was unreliable']\",\"question\":\"When cross-validation of a set of predictors results in great shrinkage of an initially encouraging multiple correlation coefficients, it is most likely chat\",\"subject\":\"professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}