{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"74b8266d-611c-564f-a014-72f773d93981","task_key":"default--test--1658","task_revision_id":"1","upstream_id":"1658","short_description":"A woman was seven months pregnant with her first child. One afternoon, the woman…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['recover, because she suffered a physical injury, rather than just emotional harm.', 'not recover, because the gang member was unaware that the woman was watching.', \\\"not recover, because the woman was not the intended target of the gang member's act.\\\", 'not recover, because the woman voluntarily assumed the risk by watching the crime.', \\\"not recover, because the miscarriage cannot be directly linked to the gang member's act.\\\", 'recover, because the gang member acted intentionally in shooting the victim.', 'not recover, because the woman was not within the zone of danger.', \\\"recover, because the gang member's act was a direct cause of her miscarriage.\\\", \\\"recover, because the gang member's act was unlawful and caused harm.\\\"]\",\"question\":\"A woman was seven months pregnant with her first child. One afternoon, the woman was in the kitchen of her home preparing a snack when she glanced out the window and saw a horrible crime being committed. The woman watched in horror as a local gang member pulled out a handgun and used it to shoot the victim, a member of a rival gang. The gang member did not know anyone was witnessing his act of shooting the victim. The woman watched as the victim collapsed and fell dead on her neighbor's front lawn. As a result of her shock from viewing this horrible incident, the woman suffered a miscarriage. If the woman asserts a claim against the local gang member who shot the victim, the woman will\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}