{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"42ae485e-745d-56f3-b31a-21f57433a099","task_key":"default--test--5664","task_revision_id":"1","upstream_id":"5664","short_description":"In what ways might the effects of chemical weapons affect the decision to deploy…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['Wind, temperature and terrain can affect the lethality and persistence of an agent and is dependent on the extensity and degree of contact with the target.', 'The color and design of chemical weapons influence the decision to use them.', 'Using chemical weapons can improve the morale of soldiers.', 'Chemical weapons can increase the yield of crops.']\",\"question\":\"In what ways might the effects of chemical weapons affect the decision to deploy them against a target?\",\"src\":\"ori_mmlu-security_studies\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}