{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"drop","formal_name":"DROP","introduction":"DROP is a reading-comprehension benchmark whose questions require discrete operations over the paragraph — addition, subtraction, comparison, counting, sorting. It is built so that extracting a span cannot answer them.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/ucinlp/drop","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3a7235a8-3664-53ff-98e3-37508f745d66","task_key":"default--validation--b6bc818a~2d878d~2d4dd0~2d8101~2d87f628b10432","task_revision_id":"1","upstream_id":"b6bc818a-878d-4dd0-8101-87f628b10432","short_description":"How many yards was the shortest touchdown pass?","config":"default","split":"validation","body":"{\"passage\":\" Starting lineups: HOUSTON - OFFENSE: QB Matt Schaub, LT Duane Brown, LG Wade Smith, C Chris Myers, RG Mike Brisiel, RT Eric Winston, WR Andre Johnson, RB Arian Foster, FB Vonta Leach, TE Joel Dreessen, WR Kevin Walter. HOUSTON - DEFENSE: LDE Antonio Smith, DT Shaun Cody, DT Amobi Okoye, RDE Mario Williams, SLB Kevin Bentley, MLB Brian Cushing, WLB Zac Diles, CB Glover Quinn, CB Kareem Jackson, SS Bernard Pollard, FS Eugene Wilson. The Texans' eighth game was an AFC duel with the Chargers. In the first quarter the Texans took the early lead as RB Arian Foster got an 8-yard TD run. The Chargers replied with QB Philip Rivers making a 55-yard TD pass to WR Seyi Ajirotutu. The Texans led again with kicker Neil Rackers nailing a 27-yard field goal, but fell behind after Rivers got an 11-yard TD pass to TE Randy McMichael. They took the lead in the second quarter as Foster made a 2-yard TD run, followed by Rackers hitting a 21 and a 25-yard field goal. The Texans lost the lead with Rivers finding McMichael again on a 12-yard TD pass. This was followed in the 4th quarter by Rivers' 28-yard TD pass to Ajirotutu (With a successful 2-point conversion as FB Mike Tolbert ran to the endzone).\",\"question\":\"How many yards was the shortest touchdown pass?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/ucinlp/drop","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}