{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b8d7f3c5-1ced-5230-9087-d6831552817a","task_key":"default--test--7092","task_revision_id":"1","upstream_id":"7092","short_description":"Why are slight differences in growth rate percentages so important?","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['They signify shifts in currency exchange rates', 'They are indicative of immediate changes in consumer confidence', 'They determine the exact size of the workforce', 'They indicate economic stability', 'They show population change', 'They indicate the rate of technological innovation', 'They can reflect large differences in potential output', \\\"They represent the country's GDP\\\", 'They are directly correlated with changes in national education levels', 'They measure the precise impact of foreign investment']\",\"question\":\"Why are slight differences in growth rate percentages so important?\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}