{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ad5e0670-637b-5698-8dbb-25e6eeb65746","task_key":"default--test--6747","task_revision_id":"1","upstream_id":"6747","short_description":"The crown of a very carious tooth suddenly fractures while the patient is…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['right lung because the right main bronchus is wider and more vertical than the left.', 'right lung because the fragment is more likely to be attracted to the right side of the body.', 'neither lung as the fragment would likely be caught in the throat.', 'left lung because the fragment is more likely to be attracted to the left side of the body.', 'left lung because the left main bronchus is wider and more vertical than the right.', 'right lung or left lung because the two main bronchi are of equal size and at the same angulation.', 'neither lung as the fragment would likely be expelled through coughing.', 'stomach because the fragment is likely to be swallowed instead of inhaled.', 'right or left lung because there are no factors predisposing to the fragment going in one particular direction.', 'esophagus because it is more likely for fragments to enter during swallowing.']\",\"question\":\"The crown of a very carious tooth suddenly fractures while the patient is standing eating at a take-away food outlet and a fragment is inhaled. It is more likely to enter the\",\"src\":\"ori_mmlu-anatomy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}