{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"76934046-bcae-5879-b504-fbc2c94bb64f","task_key":"default--test--7311","task_revision_id":"1","upstream_id":"7311","short_description":"Which of the following is correct concerning logit and probit models?","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['The logit model can result in too many observations falling exactly at the midpoint', 'They use a different method of transforming the model so that the probabilities lie between zero and one', 'The logit and probit models are identical in every aspect', 'For the logit model, the marginal effect of a change in one of the explanatory variables is simply the estimate of the parameter attached to that variable, whereas this is not the case for the probit model', 'Both logit and probit models fail to account for the possibility of error in the dependent variable.', 'The logit model can result in too many observations falling at exactly zero or exactly one', 'The logit model is less flexible in terms of shape and is always symmetric, while the probit model can accommodate asymmetry', 'For the probit model, the marginal effect of a change in one of the explanatory variables is simply the estimate of the parameter attached to that variable, whereas this is not the case for the logit model', 'The probit model is based on a cumulative logistic function']\",\"question\":\"Which of the following is correct concerning logit and probit models?\",\"src\":\"ori_mmlu-econometrics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}