{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"drop","formal_name":"DROP","introduction":"DROP is a reading-comprehension benchmark whose questions require discrete operations over the paragraph — addition, subtraction, comparison, counting, sorting. It is built so that extracting a span cannot answer them.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/ucinlp/drop","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0aa6c3a2-3958-5c71-b648-c7c3c17c9ef5","task_key":"default--validation--85715cf3~2d9e6b~2d403c~2dbb2d~2d55431dc127bc","task_revision_id":"1","upstream_id":"85715cf3-9e6b-403c-bb2d-55431dc127bc","short_description":"What quarter did the Giants get their first touchdown?","config":"default","split":"validation","body":"{\"passage\":\" A power outage that disrupted play in the third quarter served as a fitting metaphor for the Giants' general lack of power on the field this night. Smith was sidelined by a torn pectoral muscle suffered during practice, and backup receiver Ramses Barden saw his season come to an end during this game by way of a torn Achilles tendon. Former Giant Jason Garrett was making his head coaching debut for a Cowboys team revitalized by the firing of head coach Wade Phillips one week earlier. The Dallas defense held the Giants to just 6 points in the first half, aided by cornerback Bryan McCann's 101-yard \\\"pick 6\\\" from his own end zone. In a dimly lit third quarter, after a bank of lights went dark, Felix Jones extended the Cowboys' lead to 20 points on a 71-yard touchdown reception. Only after a total blackout caused an eight-minute play stoppage did Manning finally put the Giants' first touchdown on the board, in the form of a 5-yard pass to Manningham. The teams continued to trade touchdowns; a 24-yard pass from Kitna to Austin was followed by a 35-yard reception by Boss. But the Giants' turnover problem resurfaced in the fourth quarter, where a fumble and an interception ended up costing them any chance at a comeback.\",\"question\":\"What quarter did the Giants get their first touchdown?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/ucinlp/drop","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}