{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"24c2b0e3-5d25-5497-8303-88021cb51e24","task_key":"default--test--1906","task_revision_id":"1","upstream_id":"1906","short_description":"Landowner's property was landlocked. He had to go to court to ask the court to…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['The easement is terminated because it cannot exceed five years in length, without being formally renewed by both parties.', 'The easement is terminated because the landowner did not pay for its use.', 'The easement continues because it was granted by the court.', 'The holder of the easement has a right to insist on its uninterrupted continuance.', 'The easement should be modified, not terminated, because the new route may not always be available.', 'The equities favor the termination of the easement where another route is available.', 'The court will most likely rule in favor of the landowner as he has used the easement for over a decade.', 'The easement will continue until the holder of the easement chooses to release it.', 'The easement continues because its purpose of access to the property still exists.']\",\"question\":\"Landowner's property was landlocked. He had to go to court to ask the court to compel the adjacent landowner to give him an easement of necessity for ingress and egress to the highway. The court granted the easement of necessity to Landowner. Over a decade later, another access route to the highway opened up through public improvements. The adjacent landowner sued to stop Landowner from continuing to use his land for ingress and egress. What will the court most likely rule in this case?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}