{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3e40425a-76e2-5e09-bbea-f99649fd5bbb","task_key":"default--test--5269","task_revision_id":"1","upstream_id":"5269","short_description":"Which of these is true of equity but not of bonds?","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['They can be issued by firms.', 'They can be traded on the secondary market.', 'They have a fixed interest rate.', 'They cannot be traded on the secondary market.', 'Their payouts are exempt from corporation tax.', 'They are considered a form of debt.', 'Their payouts (i.e. dividend or coupons) are made after corporation tax has already been deducted.', 'They are issued by government entities.', 'Their payouts (i.e. dividends or coupons) are made before corporation tax has been deducted.']\",\"question\":\"Which of these is true of equity but not of bonds?\",\"src\":\"ori_mmlu-professional_accounting\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}