{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cd2b4965-85c0-53fc-8fe4-f2cbd16032c9","task_key":"default--test--1701","task_revision_id":"1","upstream_id":"1701","short_description":"At trial in an action for personal injuries suffered in a traffic accident, the…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['The defendant recently filed an action against the witness for breach of contract.', 'The witness has been involved in similar traffic accidents before.', 'The witness and the defendant have a personal dispute unrelated to the traffic accident.', 'In letters to prospective employers, the witness has described the defendant as very honest and dependable.', 'The witness has been falsifying his income tax returns.', 'The witness has previously perjured himself in court.', \\\"The plaintiff's injuries were not as serious as the plaintiff is claiming.\\\", 'The defendant has a past record of traffic violations.', 'The witness was under the influence of alcohol at the time of the accident.', 'The defendant is planning to sue the witness for defamation.']\",\"question\":\"At trial in an action for personal injuries suffered in a traffic accident, the plaintiff first calls the defendant as an adverse party. The plaintiff then calls a witness who was a passenger in the plaintiff's car but who also happens to be the defendant's former employer. On direct examination, the witness testifies to how the accident occurred and also expresses his opinion that the defendant is not a truthful person. Which one of the following areas of questioning is most likely to be held beyond the proper scope of cross-examination?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}