{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"52d14588-abb1-55da-9d05-1c7cc46ea9f2","task_key":"default--test--6616","task_revision_id":"1","upstream_id":"6616","short_description":"A 68-year-old female has a history of breast cancer that has recurred twice in…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['direct soft tissue technique', 'high velocity, low amplitude', 'high velocity, high amplitude', 'indirect myofascial release', 'low velocity, low amplitude', 'isotonic exercise therapy', 'mid velocity, mid amplitude', 'static stretching technique', 'direct myofascial release', 'isotonic resistance therapy']\",\"question\":\"A 68-year-old female has a history of breast cancer that has recurred twice in the past 5 years. She has undergone both radiation and chemotherapy for metastatic carcinoma to the bone. She presents to the office today to ask for help in dealing with midthoracic back pain to enable her to use less analgesic agents. The most appropriate osteopathic manipulative technique to treat this patient's spine is\",\"src\":\"ori_mmlu-professional_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}