{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1bfcb053-9ce1-56ad-8e00-b7ed9063263d","task_key":"default--test--1116","task_revision_id":"1","upstream_id":"1116","short_description":"A man works at a tire warehouse, where his job duties consist of loading tires…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['admissible as a statement for the purposes of medical diagnosis.', 'admissible as a statement made in the course of treatment.', 'inadmissible as a violation of the physicianpatient privilege.', 'admissible as it is a statement of fact.', 'admissible as expert testimony.', 'admissible as it is a statement made in anticipation of litigation.', 'inadmissible as it is a speculative statement.', 'inadmissible as it is a biased statement.', 'inadmissible as hearsay not within any recognized exception.', 'inadmissible as it is a self-serving statement.']\",\"question\":\"A man works at a tire warehouse, where his job duties consist of loading tires onto trucks. The job is exhausting, and the warehouse is always short of personnel. One day, while loading a heavy tractor tire onto a truck by himself, he experienced a sharp and excruciating pain in his lower back. He did not report the incident to his employer, but continued to work. A week later, after work, he went to the doctor for treatment. The man then sues his employer, alleging that the employer failed to provide him with adequate assistance, safety equipment, and training to load heavy tractor tires, thereby contributing to his injury. The employer alleges that the man is merely trying to obtain compensation for an old back injury, unrelated to his employment. At trial, the man's doctor testified that the man told her that his back began hurting while he was loading the tire on the truck. The doctor's testimony is\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}