{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"drop","formal_name":"DROP","introduction":"DROP is a reading-comprehension benchmark whose questions require discrete operations over the paragraph — addition, subtraction, comparison, counting, sorting. It is built so that extracting a span cannot answer them.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/ucinlp/drop","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"13e397ad-ebba-5279-a7d2-0d790e4f475b","task_key":"default--validation--dc2cba91~2db1a7~2d4448~2d8d07~2d960e33189847","task_revision_id":"1","upstream_id":"dc2cba91-b1a7-4448-8d07-960e33189847","short_description":"How many catches did Santonio Holmes have?","config":"default","split":"validation","body":"{\"passage\":\" In a matchup of fellow rookie quarterbacks Geno Smith and E.J. Manuel, the game started off well for the Jets as Geno Smith ran in the end zone for a TD, and accounted for another on a long strike to Stephen Hill. The defense was stingy, holding off E.J. Manuel and the Bills offense throughout the game leading into the third quarter, and Nick Folk delivered two successful field goals to create a 20-6 cushion at the beginning of the 3rd quarter. However, it was to be acknowledged that the Jets committed a team-record 20 penalties for 168 yards, which helped the Bills get back into the game by the 4th quarter. Eventually, Geno Smith stepped up under pressure and delivered a game-winning, 69-yard TD to Santonio Holmes. As a result, the Jets proved that they can win in the face of adversity. Bilal Powell had a good day, rushing 27 times for 149 rushing yards.\",\"question\":\"How many catches did Santonio Holmes have?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/ucinlp/drop","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}