{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"aa750fdd-6779-5879-ad88-db1522f78555","task_key":"default--test--6621","task_revision_id":"1","upstream_id":"6621","short_description":"Which of the following statements is correct?","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['The digestibility and bioavailability of isoflavones in soya food products are not changed by processing.', 'The bioavailability of isoflavones in soya food products increases with processing.', 'ß-Carotene and lycopene can both act as provitamin A.', 'Carotenoids and flavonoids are both types of carotenoids.', 'The flavonoid class comprises flavonoids and isoflavonoids.', 'All carotenoids can act as provitamin A.', 'Isoflavones and flavones are interchangeable terms.', 'Anthocyanins and flavonoids are the same class of compounds.', 'Polyphenols and flavonoids are synonymous.', 'Carotenoids and flavonoids both belong to the polyphenol class.']\",\"question\":\"Which of the following statements is correct?\\n\",\"src\":\"ori_mmlu-nutrition\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}