{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"drop","formal_name":"DROP","introduction":"DROP is a reading-comprehension benchmark whose questions require discrete operations over the paragraph — addition, subtraction, comparison, counting, sorting. It is built so that extracting a span cannot answer them.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/ucinlp/drop","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1b699910-6ac9-5ea2-a878-928d62e911a0","task_key":"default--validation--4cc640bb~2db25d~2d4fd8~2d9c68~2dade91cc720ad","task_revision_id":"1","upstream_id":"4cc640bb-b25d-4fd8-9c68-ade91cc720ad","short_description":"Which team returned an interception for a touchdown?","config":"default","split":"validation","body":"{\"passage\":\" Coming off their win over the Raiders the 49ers flew to Bank of America Stadium for an NFC duel with the Panthers. In the first quarter the 49ers took the lead as QB Alex Smith got a 1-yard TD pass to TE Vernon Davis. The lead was cut when kicker John Kasay nailed a 47-yard field goal. The 49ers scored with kicker Joe Nedney making a 24-yard field goal. The Panthers tied the game with QB Matt Moore completing an 18-yard TD pass to WR David Gettis. The 49ers fell behind in the third quarter with Kasay booting a 55-yard field goal. They eventually got the lead back in the fourth quarter with Nedney hitting a 38-yard field goal, and with DE Ray McDonald returning an interception 31 yards for a touchdown. However, the Panthers tied the game with Moore finding Gettis again on a 23-yarrd TD pass. Following an interception by 49ers backup quarterback David Carr, Kasay successfully booted a 37-yard field goal to give the 49ers the loss.\",\"question\":\"Which team returned an interception for a touchdown?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/ucinlp/drop","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}