{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"58a0bef9-5d76-5013-8fcf-50045bc08574","task_key":"default--test--1103","task_revision_id":"1","upstream_id":"1103","short_description":"A dance student worked very hard to strengthen her legs, including performing…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"[\\\"No, because it is hearsay, the janitor's statement is not a public record.\\\", 'Yes, because it is an exception to hearsay as a recorded recollection.', 'Yes, because it is an exception to hearsay as a statement made for medical diagnosis or treatment.', 'Yes, because it is as an exception to hearsay as a spontaneous declaration to an opponent-party.', 'No, because it is hearsay and the building owner is not a credible source.', 'Yes, because it is an exception to hearsay as a statement of then-existing mental, emotional, or physical condition.', 'No, because it is hearsay within hearsay, and there are no hearsay exceptions that apply.', 'Yes, because the building owner can testify and be questioned about what the janitor told him.', 'No, because it is irrelevant and immaterial.', \\\"No, because it is hearsay and there are no exceptions that would allow for the janitor's statement to be admissible.\\\"]\",\"question\":\"A dance student worked very hard to strengthen her legs, including performing additional activities such as leaping and bounding up multiple flights of stairs in her apartment building. While she was performing her exercises, one of the janitors in the building warned her that \\\"she was bound to get hurt if she keeps it up, because there are problems with the floor.\\\" After falling and breaking a leg due to a loose floorboard, she brought suit against the building owner. The building owner wants to testify that: \\\"The janitor told me 'I told her that running too fast is dangerous and she was bound to get hurt.' \\\". Is the building owner's statement likely to be held admissible?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}