{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"11dd6e53-cf4a-5e68-91df-d2d8688fb105","task_key":"default--test--9720","task_revision_id":"1","upstream_id":"9720","short_description":"Which configuration of battery and resistors will create a circuit with the…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['A high voltage battery with resistors arranged in parallel', 'A high voltage battery with resistors arranged in series', 'A low voltage battery with resistors arranged in parallel', 'Multiple low voltage batteries with resistors arranged in series', 'A low voltage battery with resistors arranged in series', 'A medium voltage battery with resistors arranged in parallel', 'A medium voltage battery with resistors arranged in series', 'A low voltage battery with no resistors', 'Multiple high voltage batteries with resistors arranged in parallel', 'A high voltage battery with no resistors']\",\"question\":\"Which configuration of battery and resistors will create a circuit with the greatest current?\",\"src\":\"ori_mmlu-high_school_physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}