{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bdf0d2de-4142-5bcb-8866-b48de6c369e6","task_key":"default--test--3191","task_revision_id":"1","upstream_id":"3191","short_description":"A biology student from Nebraska, vacationing at a Virginia beach, observes that…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"[\\\"The crab's activity cycle is influenced by barometric pressure changes\\\", \\\"The crab's activity cycle is synchronized with the coastal tides they were originally exposed to\\\", \\\"The crab's activity cycle is determined by the moon's gravitational pull\\\", \\\"The crab's activity is random and shows no particular cycle\\\", \\\"The crab's activity cycle is affected by the local environment\\\", \\\"The crab's activity cycle is determined by artificial lighting conditions\\\", \\\"The crab's activity cycle adapts to the nearest body of freshwater\\\", \\\"The crab's activity cycle is influenced by the presence of sand\\\", \\\"The crab's activity cycle is determined by the water level\\\", \\\"The crab's activity cycle is set by the day-night cycle and unaffected by tidal patterns\\\"]\",\"question\":\"A biology student from Nebraska, vacationing at a Virginia beach, observes that the level of activity of a fiddler crab changes with the cycle of the tides. When he brings some crabs back with him to Nebraska, he notices that a similar activity cycle is shown, yet there are no ocean beaches for miles. Explain.\",\"src\":\"stemez-Biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}