{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c79f7c7f-d670-51e6-a036-c208a3327d94","task_key":"default--test--6890","task_revision_id":"1","upstream_id":"6890","short_description":"What is the main contrast between the short-run and long-run Phillips curve?","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['In the short run there is a negative relationship between inflation and unemployment and in the long run the relationship is positive.', 'In the short run there is a constant relationship between inflation and unemployment and in the long run the relationship is positive.', 'In the short run there is a constant relationship between inflation and unemployment and in the long run the relationship is negative.', 'In the short run there is a negative relationship between inflation and unemployment and in the long run the relationship is constant.', 'In the short run there is a positive relationship between inflation and unemployment, in the long run the relationship is non-existent.', 'In the short run, inflation and unemployment are unrelated, while in the long run, there is a positive relationship.', 'In the short run there is a positive relationship between inflation and unemployment and in the long run the relationship is constant.', 'In the short run there is a positive relationship between inflation and unemployment and in the long run the relationship is negative.', 'In the short run, inflation and unemployment are unrelated, while in the long run, there is a negative relationship.']\",\"question\":\"What is the main contrast between the short-run and long-run Phillips curve?\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}