{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"drop","formal_name":"DROP","introduction":"DROP is a reading-comprehension benchmark whose questions require discrete operations over the paragraph — addition, subtraction, comparison, counting, sorting. It is built so that extracting a span cannot answer them.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/ucinlp/drop","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6abbc418-2c5b-51b1-a6cf-873e8c352db9","task_key":"default--validation--96a27b0b~2dddcf~2d42c8~2d9651~2d6074c40a9d4f","task_revision_id":"1","upstream_id":"96a27b0b-ddcf-42c8-9651-6074c40a9d4f","short_description":"Whose second quarter touchdown pass was longer Roethlisberger or Collins?","config":"default","split":"validation","body":"{\"passage\":\" The Titans began their season at Heinz Field for the annual kickoff game against the defending Super Bowl champions, the Pittsburgh Steelers.  After a scoreless first quarter, Tennessee trailed near the end of the second quarter as Steelers quarterback Ben Roethlisberger completed a 34-yard touchdown pass to wide receiver Santonio Holmes.  Afterwards, the Titans would tie the game as quarterback Kerry Collins completed a 16-yard touchdown pass to wide receiver Justin Gage. After a scoreless third quarter, Tennessee took the lead as kicker Rob Bironas got a 45-yard field goal, while Pittsburgh kicker Jeff Reed made a 32-yard field goal to tie the game.  In overtime, the Steelers got the win as Reed nailed a game-winning 33-yard field goal. With the loss, the Titans started the season out at 0-1.\",\"question\":\"Whose second quarter touchdown pass was longer Roethlisberger or Collins?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/ucinlp/drop","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}