{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"be264ad0-e3f9-525a-b7f7-52eb06bc0ba3","task_key":"default--test--2869","task_revision_id":"1","upstream_id":"2869","short_description":"A DNA strand with the sequence 5′ CGA TTG 3′ would be complementary to the…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['5′ TCG AAT 3′', '5′ GTT AGC 3′', '5′ GCU AAC 3′', '5′ GTT CGA 3′', '5′ TTA CGC 3′', '5′ AGC TTA 3′', '5′ CGA TTG 3′', '5′ GCT AAC 3′', '5′ CAA TCG 3′', '5′ ACG TTA 3′']\",\"question\":\"A DNA strand with the sequence 5′ CGA TTG 3′ would be complementary to the sequence\",\"src\":\"ori_mmlu-college_biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}