{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c786fec1-b8a7-5fc7-868f-15db160fdf53","task_key":"default--test--2271","task_revision_id":"1","upstream_id":"2271","short_description":"When cross-validation of a set of predictors results in great shrinkage of an…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['the criterion variable was unreliable', 'the criterion variable was overvalued', 'the selection of weights for the predictors capitalized on chance', 'the number of independent variables used in the first study was too small', 'the model was not properly validated', 'the predictor variables were not independent', 'the predictor variables were too diverse', 'none of the predictor variables was sufficiency reliable', 'the selection of weights for the predictors was biased']\",\"question\":\"When cross-validation of a set of predictors results in great shrinkage of an initially encouraging multiple correlation coefficients, it is most likely chat\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}