{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e7c0234e-02e9-588e-b6db-23f719542afd","task_key":"default--test--1115","task_revision_id":"1","upstream_id":"1115","short_description":"A man is the owner of record of a parcel of land designated as \"A. \" It is the…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['easement by necessity.', 'profit à prendre.', 'easement by implication.', 'easement in gross.', 'express easement.', 'license.', 'equitable servitude.', 'easement appurtenant.', 'natural easement.', 'prescriptive easement.']\",\"question\":\"A man is the owner of record of a parcel of land designated as \\\"A. \\\" It is the highest of three neighboring properties on a mountain. The man's friend is the owner of parcel \\\"B,\\\" which is situated lower than \\\"A\\\" on the mountainside. The man's cousin owns parcel \\\"C,\\\" which lies below parcels \\\"A\\\" and \\\"B\\\" on the mountain slope. In 2000, the man who originally owned all three parcels of land constructed a private drainage system. This system consisted of an underground pipeline that extended across all three parcels of land. Sewage from parcel \\\"A\\\" drained through the system to a municipal sewer, which was located at the bottom of the mountain. In 2005, the man sold parcel \\\"B\\\" to his friend. The following year, the cousin purchased parcel \\\"C\\\" from the man. The deeds to the friend and the cousin did not contain any mention of the man's drainage system. Located on each of the parcels are large ranch-style homes occupied by the respective owners. From the standpoint of the friend and the cousin, thedrainage system would most likely be defined as a(an)\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}