{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"83837fdb-6fd1-5961-b446-65381f32a262","task_key":"default--test--1348","task_revision_id":"1","upstream_id":"1348","short_description":"Which statement best describes the profit sharing relationship of a general…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['Partners share in proportion to their experience and expertise.', 'Partners share in proportion to their contributions to the capital and assets of the partnership.', 'Partners share in proportion to their voting percentage.', 'Partners cannot share until they unanimously agree upon a distribution.', 'Partners share in proportion to their role in the partnership.', 'Partners share according to a predetermined ratio agreed upon at the start of the partnership.', 'Partners share in proportion to the time they spend working for the partnership.', 'Partners share is determined by a third-party arbitrator.', 'Partners share equally.', 'Partners share in proportion to their individual profits.']\",\"question\":\"Which statement best describes the profit sharing relationship of a general partnership where the partners have agreed only on voting percentage and the voting shares are unequal?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}