{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fe5faafb-337c-568b-9d37-65b6f6049fad","task_key":"default--test--7231","task_revision_id":"1","upstream_id":"7231","short_description":"Imagine someone who is not looking for work because they are embarrassed in the…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['employed but seeking better employment.', 'in the labor force but not actively seeking employment.', 'in the labor force and employed.', 'in the labor force but not counted as unemployed.', 'not in the labor force but counted as unemployed.', 'not in the labor force but considered underemployed.', 'not in the labor force but seeking employment.', 'in the labor force but considered underemployed.', 'in the labor force and unemployed.', 'not in the labor force.']\",\"question\":\"Imagine someone who is not looking for work because they are embarrassed in the interview process when their inability to read is revealed. However this person would take just about any job that was offered. According to the Bureau of Labor Statistics this person is\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}