{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0cf0f589-84ab-5dae-b27b-1428ab6da8c2","task_key":"default--test--918","task_revision_id":"1","upstream_id":"918","short_description":"A woman went to a nearby supermarket late one night. She parked in the store's…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['Yes, the store knew of previous criminal activities but did not take actions to prevent them.', 'No, the store owes no special duty of care to eliminate crime on the parking lot; it was up to the police to perform that task.', 'Yes, a store has a duty to protect its customers from all forms of harm.', 'No, the store cannot be held responsible for the actions of a third party.', 'No, the store cannot predict or prevent all possible criminal activities.', 'No, the occurrence of a rape of a customer was not a foreseeable event and the store had no duty to try and discover that such a potential danger could exist.', 'Yes, a store owner is the guarantor to every customer that he or she will be protected and safe when walking in the parking lot.', 'Yes, the store failed in its duty to provide adequate security measures.', 'No, the woman should have taken measures to protect herself.', 'Yes, a business will be liable for known conditions in the building or adjoining parking areas that are a danger to the personal safety and security of its customers.']\",\"question\":\"A woman went to a nearby supermarket late one night. She parked in the store's parking lot, and while walking toward the building two masked males assaulted and raped her. The lot was poorly lit, had no security personnel, and was frequented by gang members. The police had several incident reports in the past 12 months documenting prior criminal activities, including some personal robberies and assaults, in the lot. The store itself had even more reports than the police. The woman sued the store for her injuries, claiming that it breached its duty to make the premises reasonably safe for its customers. Will she likely prevail?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}