{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ff178669-2b5b-557c-84fb-67ddcaa3000c","task_key":"default--test--972","task_revision_id":"1","upstream_id":"972","short_description":"Which of the following statements is correct concerning the \"reasonable person\"…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['The reasonable person standard applies only to people with a high level of education.', 'The reasonable person standard applies only to people without disabilities.', 'The reasonable person standard varies from person to person.', 'The reasonable person standard requires that a person act as would the average person in the community.', 'A person with a mental disability must act as would a person with the same mental disability.', 'A person with a physical disability must act as would a reasonable person with the same disability.', \\\"The reasonable person standard focuses on the defendant's subjective mental state rather than on the defendant's behavior\\\", \\\"The reasonable person standard is based solely on the defendant's age.\\\", 'A person with a mental disability is not subject to the reasonable person standard.']\",\"question\":\"Which of the following statements is correct concerning the \\\"reasonable person\\\" standard in tort law?\",\"src\":\"ori_mmlu-jurisprudence\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}