{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5fc6d795-bf5a-5073-8914-5ea3dbd0f8e7","task_key":"default--test--2076","task_revision_id":"1","upstream_id":"2076","short_description":"When it comes to STATISTICAL DECISION-MAKING, rejecting a false null is the goal…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['null hypothesis is accepted, but it is false', 'the level of significance is set by the researcher at .01 or .05', 'graphically→ white is retention area and black is rejected area', 'probability of making a Beta Level error', 'null hypothesis is rejected, but it is true', 'null hypothesis is accepted, and it is true', 'null hypothesis is not rejected, but it is false', 'the level of significance is set by the researcher at .10 or .20', 'graphically→ black is retention area and white is rejected area']\",\"question\":\"When it comes to STATISTICAL DECISION-MAKING, rejecting a false null is the goal of most researches. From your knowledge about TYPE I ERROR, which statement out of the following ones is about it?\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}