{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"81db8627-c7d6-5287-912e-96c01dbf9294","task_key":"default--test--9629","task_revision_id":"1","upstream_id":"9629","short_description":"A circuit consists of a battery and a light bulb. At first, the circuit is…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['The net charge depends on the power of the battery.', 'The net charge has become more positive.', 'The net charge has halved.', 'The net charge has doubled.', 'The net charge has completely discharged.', 'The net charge has become neutral.', 'The net charge has become zero.', 'The net charge has not changed.', 'Whether the net charge becomes more positive or more negative depends on the initial net charge residing on the circuit elements before the bulb was lit.', 'The net charge has become more negative.']\",\"question\":\"A circuit consists of a battery and a light bulb. At first, the circuit is disconnected. Then, the circuit is connected, and the light bulb lights. After the light bulb has been lit for a few moments, how has the net charge residing on the circuit elements changed?\",\"src\":\"ori_mmlu-high_school_physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}