{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"356d254b-ab38-5ecd-ae89-afe5553f293e","task_key":"default--test--2710","task_revision_id":"1","upstream_id":"2710","short_description":"Which of the following is NOT true about the difference between traditional…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['Program evaluation uses unique quantitative and qualitative methodologies to collect data. These methods are different from those that are used in traditional research.', 'Traditional research and program evaluation have different aims. Traditional research aims to test existing theories and discover generalizable knowledge, while program evaluation aims to collect information important for timely decision-making.', 'Both traditional research and program evaluation may incorporate rigorous and systematic ways of collecting data in order to address their aims.', 'Program evaluation questions are often identified through the needs of a program’s staff and stakeholders, while traditional research questions are identified by researchers using previous, existing research.']\",\"question\":\"Which of the following is NOT true about the difference between traditional research and program evaluation?\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}