{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"36a8a90d-465a-583c-a0d2-8d9a26236409","task_key":"default--test--1958","task_revision_id":"1","upstream_id":"1958","short_description":"A man and a woman are tennis partners who play regularly. Last January, the man…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['inadmissible as it is prejudicial to the man.', 'inadmissible as hearsay.', 'inadmissible as it is irrelevant to the case.', 'admissible as a statement of physical condition.', 'inadmissible as it is not related to the accident.', 'admissible as a statement of bodily condition.', 'admissible as a proper lay opinion.', 'admissible as a prior inconsistent statement.', 'admissible as an exception to the hearsay rule.', 'inadmissible as opinion testimony.']\",\"question\":\"A man and a woman are tennis partners who play regularly. Last January, the man slipped and fell while restocking grocery shelves at work. The man is suing the grocery store for damages, claiming that he suffers from pain in his back and legs as a result of the accident. At trial, the woman testified that five days prior to the accident, the man told her, \\\"I can't complete the tennis match because of pains in my back and legs. \\\"The trial judge should rule this testimony\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}