{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"184e57a3-5caa-5e3a-ad94-ac9c60d591cb","task_key":"all--test--184e57a3-5caa-5e3a-ad94-ac9c60d591cb","task_revision_id":"1","upstream_id":"","short_description":"African lungfish often live in small, stagnant, freshwater pools. With reduced…","config":"all","split":"test","body":"{\"choices\":\"['Urea is insoluble in water and sinks to the bottom of the small pools. Thus the poisonous substance is isolated from the fish.', 'Ammonia, which is highly toxic, would endanger lungfish that inhabit the small pools.', 'Other fish in the small pools can use urea as an energy source.', 'The highly toxic urea makes the pool uninhabitable to competitors.']\",\"question\":\"African lungfish often live in small, stagnant, freshwater pools. With reduced gills, they differ from other fish in that they breathe air. They also differ in that they release the nitrogen waste urea instead of ammonia. Which of the following statements correctly explains how this is an advantage for lungfish?\",\"subject\":\"high_school_biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}