{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ef6b1b44-f8e3-5593-aef8-02f27917132c","task_key":"default--test--9722","task_revision_id":"1","upstream_id":"9722","short_description":"Experimenter A uses a very small test charge qo, and experimenter B uses a test…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['greater than the field found by B', 'four times greater than the field found by B', 'either greater or less than the field found by B, depending on the accelerations of the test charges', 'the same as the field found by B', 'half the size of the field found by B', 'infinitely larger than the field found by B', 'zero, regardless of the test charge', 'dependent on the distance between the two plates', 'less than the field found by B', 'twice as strong as the field found by B']\",\"question\":\"Experimenter A uses a very small test charge qo, and experimenter B uses a test charge 2qo to measure an electric field produced by two parallel plates. A finds a field that is\",\"src\":\"ori_mmlu-high_school_physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}