{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4c092e9f-ad7d-5b71-b5f2-a2dc8488474f","task_key":"default--test--7066","task_revision_id":"1","upstream_id":"7066","short_description":"If a Durbin Watson statistic takes a value close to zero, what will be the value…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Close to either minus one or plus one', 'Exactly zero', 'Close to minus one', 'Close to zero', 'Exactly minus one', 'Slightly greater than plus one', 'Anywhere between minus one and plus one', 'Exactly plus one', 'Slightly less than minus one', 'Close to plus one']\",\"question\":\"If a Durbin Watson statistic takes a value close to zero, what will be the value of the first order autocorrelation coefficient?\",\"src\":\"ori_mmlu-econometrics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}