{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"drop","formal_name":"DROP","introduction":"DROP is a reading-comprehension benchmark whose questions require discrete operations over the paragraph — addition, subtraction, comparison, counting, sorting. It is built so that extracting a span cannot answer them.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/ucinlp/drop","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"94862791-71d1-5f34-ad3d-97af46964c59","task_key":"default--validation--514e5b11~2d61ed~2d44af~2dbde5~2ddf9a5603c126","task_revision_id":"1","upstream_id":"514e5b11-61ed-44af-bde5-df9a5603c126","short_description":"How many passes did Kurt Warner throw?","config":"default","split":"validation","body":"{\"passage\":\" Hoping to rebound from their home loss to the Vikings, the Cardinals flew to Gillette Stadium for a Week 16 interconference duel with the New England Patriots. Arizona would trail early in the first quarter as Patriots running back LaMont Jordan got a one-yard and a three-yard touchdown run. The Cardinals' east coast struggles continued in the second quarter as quarterback Matt Cassel completed a 15-yard touchdown pass to running back Kevin Faulk and an 11-yard touchdown pass to wide receiver Wes Welker, followed by kicker Stephen Gostkowski's 38-yard field goal. In the third quarter, Arizona's deficit continued to climb as Cassel completed a 76-yard touchdown pass to wide receiver Randy Moss, followed by Gostkowski's 35- and 24-yard field goal. In the fourth quarter, New England concluded its domination with Gostkowski's 30-yard field goal. The Cardinals would then get a meaningless touchdown as quarterback Matt Leinart completed a 78-yard touchdown pass to wide receiver Larry Fitzgerald. Original starting quarterback Kurt Warner (6/18 for 30 yards) was pulled in the third quarter via coach's decision.\",\"question\":\"How many passes did Kurt Warner throw?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/ucinlp/drop","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}