{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"drop","formal_name":"DROP","introduction":"DROP is a reading-comprehension benchmark whose questions require discrete operations over the paragraph — addition, subtraction, comparison, counting, sorting. It is built so that extracting a span cannot answer them.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/ucinlp/drop","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"afc6327f-9e94-5e36-9a99-9a46c38c5700","task_key":"default--validation--d1fa3cad~2d8beb~2d4681~2da923~2d76afbd768a29","task_revision_id":"1","upstream_id":"d1fa3cad-8beb-4681-a923-76afbd768a29","short_description":"Who had the shortest Touchdown in the first quarter?","config":"default","split":"validation","body":"{\"passage\":\" Coming off their embarrassing road loss to the Bears, the Broncos went to home McAfee Coliseum for an AFC West rematch with the Oakland Raiders.  In the first quarter, Denver trailed early as QB Josh McCown completed a 15-yard TD pass to WR Tim Dwight.  The Broncos replied with RB Travis Henry getting a 4-yard TD run.  In the second quarter, Oakland regained the lead with TE Zach Miller for the only score of the period. In the third quarter, the Raiders continued their domination with kicker Sebastian Janikowski getting a 38-yard field goal, while McCown completed a 13-yard TD pass to WR Jerry Porter.  The Broncos continued to try to keep up with kicker Jason Elam getting a 29-yard and a 44-yard field goal.  In the fourth quarter, Denver tried to come back as Henry got a 3-yard TD run.  However, Oakland sealed the win with Janikowski nailing a 44-yard field goal and RB Justin Fargas getting a 5-yard TD run. In the game, Denver committed 4 turnovers (2 interceptions and 2 lost fumbles), which is tied as a season-worst for the Broncos.\",\"question\":\"Who had the shortest Touchdown in the first quarter?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/ucinlp/drop","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}