{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"85be343a-f10f-599d-81a1-baf3305bda34","task_key":"default--test--10900","task_revision_id":"1","upstream_id":"10900","short_description":"Construct a complete truth table for the following pairs of propositions. Then,…","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['Neither logically equivalent nor contradictory, but consistent', 'Both contradictory and inconsistent', 'Neither logically equivalent, contradictory, consistent, nor inconsistent', 'Both contradictory and consistent', 'Both logically equivalent and consistent', 'Logically equivalent', 'Contradictory', 'Inconsistent', 'Neither logically equivalent nor contradictory, but inconsistent', 'Both logically equivalent and contradictory']\",\"question\":\" Construct a complete truth table for the following pairs of propositions. Then, using the truth tables, determine whether the statements are logically equivalent or contradictory. If neither, determine whether they are consistent or inconsistent. Justify your answers.\\nI ≡ ~H and ~(~I ≡ H)\",\"src\":\"ori_mmlu-formal_logic\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}