{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b04f0eb7-5778-588a-bcdd-be2c09ed575d","task_key":"default--test--7699","task_revision_id":"1","upstream_id":"7699","short_description":"Biologists around the world have increased efforts at conservation. Monitoring…","config":"default","split":"test","body":"{\"category\":\"math\",\"options\":\"['The correlation coefficient is -0.57.', 'The correlation coefficient is -0.84.', 'The correlation coefficient is 0.57.', 'The correlation coefficient is 0.71.', 'The correlation coefficient is 0.84.', 'The correlation coefficient is 0.29.', 'The correlation coefficient is -1.00.', 'The correlation coefficient is 1.00.', 'The correlation coefficient is -0.29.', 'The correlation coefficient is -0.71.']\",\"question\":\"Biologists around the world have increased efforts at conservation. Monitoring wildlife populations is important so that appropriate management efforts can be implemented, but such monitoring is often difficult. One study found a cheap and easy way to estimate the number of nesting sites of terns (a type of seabird) by monitoring the number of calls heard per minute. More calls happen when there are more birds, as one would expect. In fact, it turned out that the number of calls explained 71% of the variation in the abundance of nests between breeding sites. Which of the following statements is correct about the correlation between the number of calls and the abundance of nests?\",\"src\":\"ori_mmlu-high_school_statistics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}