{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"715f24b9-a9a0-5720-87ac-5ba78e2c5465","task_key":"default--test--1685","task_revision_id":"1","upstream_id":"1685","short_description":"A landlord was the owner of a two-story dwelling house and leased it completely…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['not recover, because privity of estate does not exist between the landlord and sublessee.', 'recover, because the landlord is the original owner of the house.', 'not recover, because the landlord did not approve the sublease agreement.', 'recover, because there is privity of contract between a landlord and sublessee.', 'not recover, because there is no privity of contract between the landlord and sublessee.', 'recover, because there is both privity of estate and privity of contract between a landlord and sublessee.', 'not recover, because the original tenant is responsible for the rent, not the sublessee.', 'recover, because there is privity of estate and privity of contract between the landlord and the original tenant.', 'recover, because the landlord has a right to payment from any tenant occupying the property.', 'recover, because pnvity of estate exists between a landlord and sublessee.']\",\"question\":\"A landlord was the owner of a two-story dwelling house and leased it completely furnished to a tenant for a 10-year period. Two years after the tenant entered into the leasehold agreement with the landlord, he subleases the house to his cousin. Thereafter, the cousin defaults on the rental payments. If the landlord brings an action to recover the past rent due from the cousin, the landlord will:\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}