{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6c90f818-dc0e-5ad4-b251-d400a4ba1f8f","task_key":"default--test--572","task_revision_id":"1","upstream_id":"572","short_description":"A framework categorizes different services, which, in turn, influence the degree…","config":"default","split":"test","body":"{\"category\":\"business\",\"options\":\"['Search, belief, and confidence properties.', 'Seek, experience, and credibility properties.', 'Search, experiment, and confidence properties.', 'Search, empirical, and credence properties.', 'Sell, trust, and credence properties.', 'Search, experiment, and credence properties.', 'Search, experience, and credence properties.', 'Search, explore, and credence properties.', 'Sight, experience, and credence properties.', 'Scan, experience, and confidence properties.']\",\"question\":\"A framework categorizes different services, which, in turn, influence the degree to which market offerings can be evaluated, and three main properties are identified:\",\"src\":\"ori_mmlu-marketing\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}