{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"900066e4-bdbc-50a4-a1a3-b4c82ac6c0f2","task_key":"default--test--1211","task_revision_id":"1","upstream_id":"1211","short_description":"Some homeless people started residing on a strip of land located under a bridge.…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['No, because owner owes no duty to trespassers except if it acts with willful or wanton disregard.', 'No, because the squatters voluntarily chose to use the polluted water, thereby assuming the risk.', 'Yes, because even though they trespassed, the owner had a duty to warn because it knew that they were in danger.', 'Yes, because the owner had a duty to ensure that its property was not causing harm to others, irrespective of their legal status.', 'No, because the company was unaware that the squatters were using the polluted water for bathing and drinking.', 'No, because the squatters should have tested the water before using it.', 'Yes, because the company knowingly created a hazardous condition and failed to rectify it.', 'No, because an owner of land never has to worry about protecting the safety of trespassers.', 'Yes, because the owner was strictly liable for any injuries caused by the hazardous condition of the water in the lake.', 'Yes, because the owner negligently allowed the pollution of the water.']\",\"question\":\"Some homeless people started residing on a strip of land located under a bridge. The land was owned by a nearby chemical manufacturer. The squatters did not ask permission, and the company did not feel it was an urgent problem. The squatters used a small lake on the property for bathing and drinking water. Within a few days, two squatters died from what was later revealed to be highly polluted water coming from the company's waste discharges. The company knew that it had a waste discharge problem. The estates of the two decedents sued the company. Will they likely prevail on their tort claim despite the company's defense that the decedents were trespassers?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}