{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d4570c27-199d-574c-a747-205ee732312e","task_key":"default--test--1009","task_revision_id":"1","upstream_id":"1009","short_description":"A developer is the owner of a parcel of land in fee simple absolute (the record…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['the artist.', 'the engineer.', 'the developer.', 'the dentist.', 'all of the above.', 'the developer and the artist jointly.', 'the bartender.', 'the counselor and the engineer jointly.', 'none of the above.']\",\"question\":\"A developer is the owner of a parcel of land in fee simple absolute (the record title is also in the developer). The parcel of land is and has been unpossessed land. Assume that each person listed below as the grantee in a deed paid value and took without actual notice of any facts that would defeat her title and without knowledge of any facts that would put her on inquiry, both at the time she took her deed and at the time she recorded it. The following conveyances, each by a general warranty deed, take place in the order listed: An artist conveys to a bartender; the developer conveys to the artist; the artist conveys to a counselor; the bartender conveys to a dentist; the counselor conveys to an engineer. No deed was recorded until after the artist-to- counselor deed was executed and then before the execution of any other deed, the order of recording was as follows: developer to artist; artist to bartender; artist to counselor. The bartender-to-dentist deed and counselor-to- engineer deed were each recorded immediately after their respective executions. The owner of the parcel of land in a notice jurisdiction is\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}