{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"04bd7f4d-44af-5946-8c15-9830cfc79800","task_key":"default--test--823","task_revision_id":"1","upstream_id":"823","short_description":"___________, ___________ and __________ are all factors which can impact the…","config":"default","split":"test","body":"{\"category\":\"business\",\"options\":\"['Social dissensus, Cultural immediacy, Proximity', 'Social consensus, Temporal immediacy, Proximity', 'Temporal consensus, Social immediacy, Proximity', 'Social consensus, Spatial immediacy, Proximity', 'Proximity, Cultural dissensus, Temporal immediacy', 'Proximity, Social consensus, Temporal dissensus', 'Cultural consensus, Temporal immediacy, Proximity', 'Proximity, Temporal immediacy, Social dissensus', 'Proximity, Social dissensus, Futility of effect', 'Social dissensus, Temporal immediacy, Distance']\",\"question\":\"___________, ___________ and __________ are all factors which can impact the moral intensity\",\"src\":\"ori_mmlu-business_ethics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}