{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"drop","formal_name":"DROP","introduction":"DROP is a reading-comprehension benchmark whose questions require discrete operations over the paragraph — addition, subtraction, comparison, counting, sorting. It is built so that extracting a span cannot answer them.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/ucinlp/drop","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ceb03b6a-e8b5-596f-85fb-ea87e5247df2","task_key":"default--validation--3aa994f0~2db9aa~2d4483~2dad1e~2d307047f4c8ce","task_revision_id":"1","upstream_id":"3aa994f0-b9aa-4483-ad1e-307047f4c8ce","short_description":"Which quarterback had the shortest touchdown pass completed?","config":"default","split":"validation","body":"{\"passage\":\" Trying to snap their six-game losing streak, the 49ers flew to Qwest Field for an NFC West rematch on Monday Night Football with the Seattle Seahawks. In the first quarter, the Niners trailed early as Seahawks QB Matt Hasselbeck completed a 1-yard TD pass to TE Will Heller, along with kicker Josh Brown. In the second quarter, San Francisco's struggles continued as RB Maurice Morris gave Seattle a 6-yard TD run for the only score of the period. After a scoreless third quarter, the Seahawks sealed the win and the season-sweep with Hasselbeck completing a 10-yard TD pass to WR D.J. Hackett.\",\"question\":\"Which quarterback had the shortest touchdown pass completed?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/ucinlp/drop","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}