{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1c7a2c30-e5f2-5176-8895-7162f4130aa0","task_key":"default--test--11074","task_revision_id":"1","upstream_id":"11074","short_description":"Soft determinism entails that _____.","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['our actions are completely predetermined', 'we have no control over our actions', 'the thesis of determinism is false', 'the thesis of free will is false', 'the thesis of determinism is true', 'there are no restraints on human behavior', 'human behavior is entirely random', 'our desires are predetermined, but our actions are not']\",\"question\":\"Soft determinism entails that _____.\",\"src\":\"ori_mmlu-philosophy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}