{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f1a588cb-18bd-5e66-b9ef-e4abe4be171d","task_key":"default--test--7445","task_revision_id":"1","upstream_id":"7445","short_description":"Which one of the following is the most appropriate definition of a 99%…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['99% of the time in repeated samples, the interval would contain the estimated value of the null hypothesis', '99% of the time in repeated samples, the interval would be different from the estimated value of the parameter', '99% of the time in repeated samples, the null hypothesis will not be rejected when it was false', '1% of the time in repeated samples, the interval would not contain the true value of the parameter', '99% of the time in repeated samples, the interval would contain the true value of the null hypothesis', '99% of the time in repeated samples, the null hypothesis will be accepted when it was false', '99% of the time in repeated samples, the null hypothesis will be rejected', '99% of the time in repeated samples, the interval would contain the estimated value of the parameter', '99% of the time in repeated samples, the interval would contain the true value of the parameter', '99% of the time in repeated samples, the interval would not contain the true value of the null hypothesis']\",\"question\":\"Which one of the following is the most appropriate definition of a 99% confidence interval?\",\"src\":\"ori_mmlu-econometrics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}