{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"311882c0-1d31-5083-83bd-15bf0ed14453","task_key":"all--test--311882c0-1d31-5083-83bd-15bf0ed14453","task_revision_id":"1","upstream_id":"","short_description":"A wholesaler brought a federal diversity action against a large pharmaceutical…","config":"all","split":"test","body":"{\"choices\":\"['No, because the potential juror said that he could fairly consider the evidence in the case.', \\\"No, because the wholesaler's attorney could use a peremptory challenge to strike the potential juror.\\\", 'Yes, because other potential jurors still remain available for the jury panel.', 'Yes, because the potential juror is presumed to be biased because of his relationship to the company.']\",\"question\":\"A wholesaler brought a federal diversity action against a large pharmaceutical company for breach of contract. During jury selection, one potential juror stated that five years earlier he had been an employee of the company and still owned sev-eral hundred shares of its stock. In response to questioning from the judge, the potential juror stated that he could fairly consider the evidence in the case. The wholesaler's attorney has asked the judge to strike the potential juror for cause. Should the judge strike the potential juror for cause?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}