{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"drop","formal_name":"DROP","introduction":"DROP is a reading-comprehension benchmark whose questions require discrete operations over the paragraph — addition, subtraction, comparison, counting, sorting. It is built so that extracting a span cannot answer them.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/ucinlp/drop","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f3389ec7-d99a-5757-af83-a56d46a57eb5","task_key":"default--validation--b09f3c95~2dc28a~2d473c~2db120~2ddf9faa59e9c0","task_revision_id":"1","upstream_id":"b09f3c95-c28a-473c-b120-df9faa59e9c0","short_description":"how many points did dallas lead by?","config":"default","split":"validation","body":"{\"passage\":\" Following their second loss to the Giants, the Cowboys returned to Arlington in a match against the Tampa Bay Buccaneers (who were coming off a five-game winning streak). Despite both teams sharing the lead from time and time again, the Cowboys were able to hold off the Buccaneers. With less than a minute remaining in the fourth quarter, Orlando Scandrick intercepted a pass from Buccaneers quarterback Jameis Winston (who fumbled earlier in the first half) and ended the game as a Cowboys victory. Dallas' only other highlight came early in the second quarter when Ezekiel Elliott leapt onto the Salvation Army kettle after scoring a touchdown to lead Dallas 10-3. This celebration resulted in a 15-yard unsportsmanlike conduct penalty; however, the following day, the NFL decided to not fine Elliott for the celebration. The Cowboys improved to 12-2, and with the Giants' loss to the Eagles the following Thursday night, they clinched the NFC East title and home field advantage throughout the playoffs.\",\"question\":\"how many points did dallas lead by?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/ucinlp/drop","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}