{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"29b8401e-0f19-50aa-849d-21793828ad5a","task_key":"default--test--496","task_revision_id":"1","upstream_id":"496","short_description":"Traditionally consumer rights were based on the notion of _______ or _______…","config":"default","split":"test","body":"{\"category\":\"business\",\"options\":\"['Caveat emptor, Buyer beware, Regulation, National', 'Cemptor Eaveat, Company Responsibility, Informal Codes, International', 'Caveat vendor, Company Responsibility, Regulation, National', 'Caveat vendor, Seller beware, Informal Codes, International', 'Caveat vendor, Buyer beware, Regulation, International', 'Caveat emptor, Company Responsibility, Informal Codes, International', 'Caveat emptor, Company Responsibility, Informal codes, International', 'Caveat emptor, Buyer beware, Informal codes, National', 'Cemptor Eaveat, Buyer beware, Regulation, National', 'Caveat emptor, Seller beware, Informal codes, National']\",\"question\":\"Traditionally consumer rights were based on the notion of _______ or _______ where the responsibility lay with consumer to avoid subpar products. However, this has gradually eroded due to increased ________ at a ________, EU and UN level.\",\"src\":\"ori_mmlu-business_ethics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}