{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"drop","formal_name":"DROP","introduction":"DROP is a reading-comprehension benchmark whose questions require discrete operations over the paragraph — addition, subtraction, comparison, counting, sorting. It is built so that extracting a span cannot answer them.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/ucinlp/drop","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f5485464-f143-526a-a607-bed282968fea","task_key":"default--validation--d49586dd~2dbd48~2d4452~2dbbc0~2dd5e478255e58","task_revision_id":"1","upstream_id":"d49586dd-bd48-4452-bbc0-d5e478255e58","short_description":"How many touchdowns did Reggie Bush score?","config":"default","split":"validation","body":"{\"passage\":\" The Saints opened the regular season on the road against the Cleveland Browns on September 10.  In the first half, kicker John Carney provided all of the Saints' first 9 points.  He put up a 43-yarder in the first, along with a 25 and a 21-yarder in the second quarter.  In the third quarter, the Browns offense finally got going as Cleveland quarterback Charlie Frye hooked up with TE Kellen Winslow on an 18-yard touchdown pass.  Fortunately, the Saints would respond with a 12-yard touchdown pass from quarterback Drew Brees to WR Marques Colston.  In the fourth quarter, the Browns came close with Frye getting a 1-yard touchdown run.  However, the Saints put the game away with Carney kicking a 20-yard FG to give the Saints the win. Despite not scoring a touchdown, RB Reggie Bush had a sound NFL debut, as he ran 14 times for 61 yards, caught 8 passes for 58 yards, and returned three punts for 22 Yards.  In total, he piled up 141 all-purpose yards.\",\"question\":\"How many touchdowns did Reggie Bush score?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/ucinlp/drop","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}