{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"639b87ef-2ed7-5ce7-89ca-3add8976eb34","task_key":"default--test--10786","task_revision_id":"1","upstream_id":"10786","short_description":"If a global ban would not be effective in prohibiting genetic engineering that…","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['none of the above', 'using a lottery system so everyone gets a fair chance at the benefits of genetic engineering', 'using sanctions and trade restrictions against countries that violate the ban', 'allowing genetic engineering only for medical purposes', 'promoting genetic engineering only for the wealthy', 'creating a global consensus on acceptable genetic engineering practices', 'developing a global regulatory body to monitor and control genetic engineering activities', 'implementing a global education program to convince people of the dangers of excessive genetic engineering', 'requiring genetic engineering for every newborn child', 'creating a world military that would enforce the ban with greater strength']\",\"question\":\" If a global ban would not be effective in prohibiting genetic engineering that goes beyond eliminating obvious defects, then Singer suggests which \\\"bolder\\\" strategy?\",\"src\":\"ori_mmlu-moral_disputes\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}