{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c1f3478d-4698-58ed-ac21-9dd8800680e6","task_key":"default--test--2581","task_revision_id":"1","upstream_id":"2581","short_description":"When one uses a time-series design to evaluate a program, fluctuation in the…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['the outcome measure was not sensitive enough', 'a simultaneous event produced the fluctuation', 'the participants did not understand the program', 'the data was not analyzed correctly', 'the program was not implemented correctly', 'the fluctuation was due to random chance', 'participants dropped out of the group, reducing the size of the sample', 'aging effects were unduly influential', 'test—retest reliability is inadequate', 'the time-series design was flawed']\",\"question\":\"When one uses a time-series design to evaluate a program, fluctuation in the outcome slope can be accounted for by various alternative hypotheses. According to Campbell and Stanley, the most probable alternative is that\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}