{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2e0308e8-4665-583f-8328-f28762e669dc","task_key":"default--test--3425","task_revision_id":"1","upstream_id":"3425","short_description":"Suppose you discovered a new species of bioluminescent worm . How could you…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['Isolate the light-producing organ and test it in isolation from the worm', \\\"Use spectroscopy to analyze the light's wavelength signature and compare it to known bioluminescent organisms\\\", 'Place the light-producing substance in complete growth media or physically examine the light-emitting substance under a microscope.', 'Treat the worm with antibiotics to see if the light production stops', 'Conduct genetic modification to knock out suspected bioluminescent genes and observe if light production ceases', 'Perform DNA sequencing on the worm', 'Look for the presence of bioluminescent proteins in the worm', \\\"Compare the worm's bioluminescence with known bacterial bioluminescence patterns\\\", 'Observe the worm in a dark environment', \\\"Measure the intensity of light produced by the worm over time to see if it correlates with the worm's life cycle\\\"]\",\"question\":\"Suppose you discovered a new species of bioluminescent worm . How could you prove that it was the worm itself and not some contaminating bacteria that was producing the light ?\",\"src\":\"stemez-Biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}