{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7852ba22-f447-581a-82e7-248852964666","task_key":"default--test--7318","task_revision_id":"1","upstream_id":"7318","short_description":"Suppose that we wished to evaluate the factors that affected the probability…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['A linear regression model', 'A Poisson regression model', 'A probit model', 'A binary logit model', 'An exponential model', 'A multinomial logit', 'A logit model', 'A tobit model', 'An ordered logit model', 'A GARCH model']\",\"question\":\"Suppose that we wished to evaluate the factors that affected the probability that an investor would choose an equity fund rather than a bond fund or a cash investment. Which class of model would be most appropriate?\",\"src\":\"ori_mmlu-econometrics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}