{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"29b44459-8c57-5357-9517-a32c04b689da","task_key":"default--test--1514","task_revision_id":"1","upstream_id":"1514","short_description":"There may be a substantial time period between contract formation and final…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['A fully executory contract is when both parties have completed their performance.', 'A partially executed contract means that both parties have begun but not completed performance.', 'An executory contract is when one party has performed and the other party has not yet begun performance.', 'An executed contract exists when both parties have fully performed and no obligations remain.', 'A partially executed contract means that at least one party has begun performance.', 'A partially executory contract is when one party has begun performance and the other party has completed performance.', 'A fully executed contract exists when neither party has any remaining obligations.', 'A partially executed contract means that one party has completed performance while the other party has only promised.', 'A wholly executory contract is where only promises have been exchanged and there has been no performance by either party.', 'An executory contract is when both parties have yet to begin performance.']\",\"question\":\"There may be a substantial time period between contract formation and final completion of performance. Concerning these executory contracts, which of the following is the least correct?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}