{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5b58e9d7-311f-5f00-aa49-d6bd52fb2f62","task_key":"default--test--3215","task_revision_id":"1","upstream_id":"3215","short_description":"Aggressive ants live in the swollen thorns of a small tropical tree and feed on…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['Remove oil-rich bodies and measure subsequent ant density.', 'Remove leaf-feeding insects and measure subsequent ant density.', 'Add more oil-rich bodies and measure subsequent leaf damage.', 'Remove leaf-feeding insects and measure subsequent leaf damage.', 'Remove ants and measure subsequent thorn growth.', 'Add more leaf-feeding insects and measure subsequent ant density.', 'Remove the thorns and measure subsequent density of ants.', 'Remove the tree and measure subsequent ant density.', 'Remove ants and measure subsequent leaf damage.', 'Add more ants and measure subsequent leaf damage.']\",\"question\":\"Aggressive ants live in the swollen thorns of a small tropical tree and feed on oil-rich bodies that the tree produces at the tips of its leaflets. Which of the following experiments best tests the hypothesis that ants defend the plant by either killing or chasing away leaf-feeding insects?\",\"src\":\"ori_mmlu-college_biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}