{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"956df34b-fb10-5f3d-b095-cb3ee8fb0fae","task_key":"default--test--1034","task_revision_id":"1","upstream_id":"1034","short_description":"Congress enacted a statute designed to make the stabilization of rents more…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['the rental of a vacation home by an individual landlord to a family on a short-term lease.', 'the rental of a state-owned office building by a state to a privately owned business.', 'the rental of a college dormitory by a private university to its students.', 'the rental of residential property located in the District of Columbia by an individual landlord to a family of six.', 'the rental of an office building to a city by an individual landlord.']\",\"question\":\"Congress enacted a statute designed to make the stabilization of rents more effective throughout the United States. The statute will be applicable to all leasehold contracts hereafter entered into between landlords and tenants involving residential and commercial properties. The new federal statute is intended to protect tenants from spiraling rents and \\\"profiteering\\\" by landlords without curtailing the supply of rental units or depriving landlords of substantial constitutional rights. The new federal statute would be most clearly constitutional as applied to\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}