{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"drop","formal_name":"DROP","introduction":"DROP is a reading-comprehension benchmark whose questions require discrete operations over the paragraph — addition, subtraction, comparison, counting, sorting. It is built so that extracting a span cannot answer them.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/ucinlp/drop","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9e403ba9-94f7-5b92-a960-3bd5b7bdca8a","task_key":"default--validation--1571187e~2d4b41~2d4f3a~2da415~2ded35fb104ac8","task_revision_id":"1","upstream_id":"1571187e-4b41-4f3a-a415-ed35fb104ac8","short_description":"How many points were scored during the first quarter?","config":"default","split":"validation","body":"{\"passage\":\" The Titans began their season at Heinz Field for the annual kickoff game against the defending Super Bowl champions, the Pittsburgh Steelers.  After a scoreless first quarter, Tennessee trailed near the end of the second quarter as Steelers quarterback Ben Roethlisberger completed a 34-yard touchdown pass to wide receiver Santonio Holmes.  Afterwards, the Titans would tie the game as quarterback Kerry Collins completed a 16-yard touchdown pass to wide receiver Justin Gage. After a scoreless third quarter, Tennessee took the lead as kicker Rob Bironas got a 45-yard field goal, while Pittsburgh kicker Jeff Reed made a 32-yard field goal to tie the game.  In overtime, the Steelers got the win as Reed nailed a game-winning 33-yard field goal. With the loss, the Titans started the season out at 0-1.\",\"question\":\"How many points were scored during the first quarter?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/ucinlp/drop","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}