{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"boolq","formal_name":"BoolQ","introduction":"BoolQ pairs naturally occurring yes/no questions from search queries with a passage that decides them. The format is simple, but the inference it needs is consistently heavier than it looks.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-research-datasets/boolean-questions","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"14b72c0b-9c36-53c0-97e9-49a35b8e43b8","task_key":"default--validation--14b72c0b-9c36-53c0-97e9-49a35b8e43b8","task_revision_id":"1","upstream_id":"","short_description":"are rats and mice from the same family","config":"default","split":"validation","body":"{\"passage\":\"Species can be arboreal, fossorial (burrowing), or semiaquatic. Well-known rodents include mice, rats, squirrels, prairie dogs, chipmunks, porcupines, beavers, guinea pigs, hamsters, gerbils and capybaras. Other animals such as rabbits, hares, and pikas, whose incisors also grow continually, were once included with them, but are now considered to be in a separate order, the Lagomorpha. Nonetheless, Rodentia and Lagomorpha are sister groups, sharing a most recent common ancestor and forming the clade of Glires.\",\"question\":\"are rats and mice from the same family\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-research-datasets/boolean-questions","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}