{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"boolq","formal_name":"BoolQ","introduction":"BoolQ pairs naturally occurring yes/no questions from search queries with a passage that decides them. The format is simple, but the inference it needs is consistently heavier than it looks.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-research-datasets/boolean-questions","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b3c587df-e0af-5153-9334-9c83e4e9eba9","task_key":"default--validation--b3c587df-e0af-5153-9334-9c83e4e9eba9","task_revision_id":"1","upstream_id":"","short_description":"is there such thing as a blue ruby","config":"default","split":"validation","body":"{\"passage\":\"Color: In the evaluation of colored gemstones, color is the most important factor. Color divides into three components: hue, saturation and tone. Hue refers to color as we normally use the term. Transparent gemstones occur in the pure spectral hues of red, orange, yellow, green, blue, violet. In nature, there are rarely pure hues, so when speaking of the hue of a gemstone, we speak of primary and secondary and sometimes tertiary hues. Ruby is defined to be red. All other hues of the gem species corundum are called sapphire. Ruby may exhibit a range of secondary hues, including orange, purple, violet, and pink.\",\"question\":\"is there such thing as a blue ruby\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-research-datasets/boolean-questions","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}