{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d4fc5b87-852e-564d-8871-10c9ccb3c626","task_key":"default--test--2089","task_revision_id":"1","upstream_id":"2089","short_description":"Of the following, the best way to increase the amount of helping behavior within…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['decrease the number of individuals in the community', 'implement strict punishments for non-helping behavior', 'encourage competition among community members', 'diffuse the responsibility among all members of the community', 'decrease the anonymity of individuals in the community', 'increase the wealth of individuals in the community', 'decrease the self-esteem of community members', 'increase the number of individuals in the community', 'develop programs that promote self-reliance', 'initiate programs to raise the self-esteem of community members']\",\"question\":\"Of the following, the best way to increase the amount of helping behavior within a community is to\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}