{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b9c5cba8-ab3c-5e7b-8f50-9a97d7ddb2b0","task_key":"default--test--10890","task_revision_id":"1","upstream_id":"10890","short_description":"According to Sartre, if God does not exist, _____.","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['man is condemned to be free', 'moral values still exist', 'morality is defined by society', 'man does not exist', 'anything is permissible', 'there is no purpose to life', 'humans are not responsible for their actions', 'life has no meaning', 'moral values must come from nature', 'morals are subjective']\",\"question\":\"According to Sartre, if God does not exist, _____.\",\"src\":\"ori_mmlu-philosophy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}