{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3e7ce1b4-dd0f-564f-b4fa-a94af2b88312","task_key":"default--test--10664","task_revision_id":"1","upstream_id":"10664","short_description":"Statement 1| The ID3 algorithm is guaranteed to find the optimal decision tree.…","config":"default","split":"test","body":"{\"category\":\"computer science\",\"options\":\"['True, True for a given data set', 'False, True', 'True, False', 'False, False unless specific conditions are met.', 'False, True only for discrete probability distributions', 'False, False', 'True, False only for specific continuous probability distributions', 'True, True only under certain conditions', 'False, True only when the optimal decision tree is defined in a specific way', 'True, True']\",\"question\":\"Statement 1| The ID3 algorithm is guaranteed to find the optimal decision tree. Statement 2| Consider a continuous probability distribution with density f() that is nonzero everywhere. The probability of a value x is equal to f(x).\",\"src\":\"ori_mmlu-machine_learning\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}