{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"56027591-1369-5f78-8d43-009ac5633632","task_key":"default--test--2505","task_revision_id":"1","upstream_id":"2505","short_description":"When a qualitative research is done, the theory is developed from the data, not…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['Case studies are based on the assumption that the case can be generalized', 'Case studies do not rely on multiple sources of information', 'Case studies are always conducted over a short period of time', 'Case studies always require a large sample size for accurate results', 'Case studies can only be used in the field of psychology', 'Case studies are the a detailed examination of a single case', 'Case studies are used for attitude measurement, preferences, & satisfaction', 'Case studies are most useful as pilot study to identify variables that can be studies in other ways', 'Case studies do not provide in-depth understanding of a particular instance']\",\"question\":\"When a qualitative research is done, the theory is developed from the data, not before and helps refine hypotheses. From your knowledge about this type of research, which one out of the following statements about CASE STUDIES can not be correct?\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}