{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"40f6578a-da01-5f0c-aec0-35e0618b4660","task_key":"default--test--7217","task_revision_id":"1","upstream_id":"7217","short_description":"If a household chose to save 10 percent of its income and to addthree quarters…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['$3,000', '$2,250', '$4,500', '$1,500', '$900', '$1,750', '$6,000', '$2,750', '$750', '$5,250']\",\"question\":\"If a household chose to save 10 percent of its income and to addthree quarters of its saving to cash balances, what wouldbe the value of the loan contracts it demanded when itsincome totaled $30,000 per year?\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}