{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6d72ddee-897c-5a1f-a297-257cbd3758bc","task_key":"all--test--6d72ddee-897c-5a1f-a297-257cbd3758bc","task_revision_id":"1","upstream_id":"","short_description":"Construct a complete truth table for the following pairs of propositions. Then,…","config":"all","split":"test","body":"{\"choices\":\"['Logically equivalent', 'Contradictory', 'Neither logically equivalent nor contradictory, but consistent', 'Inconsistent']\",\"question\":\" Construct a complete truth table for the following pairs of propositions. Then, using the truth tables, determine whether the statements are logically equivalent or contradictory. If neither, determine whether they are consistent or inconsistent. Justify your answers.\\n~(J ∨ K) · L and (L ⊃ J) · K\",\"subject\":\"formal_logic\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}