{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a8bfeacc-af63-5301-8ff1-d083d52cf816","task_key":"default--test--7308","task_revision_id":"1","upstream_id":"7308","short_description":"Which one of the following is a disadvantage of the general to specific or \"LSE\"…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Some variables may be excluded at the first stage leading to coefficient biases', 'The final model may lack theoretical interpretation', 'The final model may be statistically inadequate', 'If the initial model is mis-specified, all subsequent steps will be invalid.']\",\"question\":\"Which one of the following is a disadvantage of the general to specific or \\\"LSE\\\" (\\\"Hendry\\\") approach to building econometric models, relative to the specific to general approach?\",\"src\":\"ori_mmlu-econometrics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}