{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"340a13f3-ab8b-5072-b5f0-9ffd78b0d1cf","task_key":"default--test--10857","task_revision_id":"1","upstream_id":"10857","short_description":"Construct a complete truth table for the following pairs of propositions. Then,…","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['Both consistent and inconsistent', 'Neither logically equivalent nor contradictory, but inconsistent', 'Contradictory', 'Both contradictory and inconsistent', 'Neither logically equivalent nor contradictory, but consistent', 'Both logically equivalent and contradictory', 'Both logically equivalent and consistent', 'Logically equivalent', 'Inconsistent', 'Neither logically equivalent, contradictory, consistent, nor inconsistent']\",\"question\":\" Construct a complete truth table for the following pairs of propositions. Then, using the truth tables, determine whether the statements are logically equivalent or contradictory. If neither, determine whether they are consistent or inconsistent. Justify your answers.\\n~O ⊃ P and O ∨ P\",\"src\":\"ori_mmlu-formal_logic\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}