{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"f4a3f1df-e413-57fd-bdd7-6837f34d3794","task_key":"default--test--76","task_revision_id":"2","upstream_id":"76","short_description":"Pine and Gilmore (1999) derive four distinct realms of experience, based on two…","config":"default","split":"test","body":"{\"category\":\"business\",\"options\":\"['Customer participation and environmental acquisition.', 'Environmental acquisition and environmental relationship.', 'Customer retention and environmental relationship.', 'Customer participation and environmental relationship.', 'Customer acquisition and customer retention.', 'Customer participation and customer relationship.', 'Customer acquisition and environmental participation.', 'Environmental participation and customer relationship.', 'Customer retention and customer relationship.', 'Customer acquisition and environmental relationship.']\",\"question\":\" Pine and Gilmore (1999) derive four distinct realms of experience, based on two dimensions. What are these dimensions?\",\"src\":\"ori_mmlu-marketing\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}