{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"drop","formal_name":"DROP","introduction":"DROP is a reading-comprehension benchmark whose questions require discrete operations over the paragraph — addition, subtraction, comparison, counting, sorting. It is built so that extracting a span cannot answer them.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/ucinlp/drop","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e5cb8ea8-b4e1-5d39-a2e2-600c0ea7cd27","task_key":"default--validation--17cbc6d3~2df844~2d4777~2da781~2d8c86516d6a07","task_revision_id":"1","upstream_id":"17cbc6d3-f844-4777-a781-8c86516d6a07","short_description":"Which team scored the only points in overtime?","config":"default","split":"validation","body":"{\"passage\":\" The Giants would lead 20-10 in the fourth quarter. However, the Jets would march down the field to tie the game with 27 seconds remaining. Randy Bullock would give the Jets the lead in overtime after he converted a 31-yard field goal. The Giants would go down the field, but a critical drop by Odell Beckham would prove them costly. Josh Brown would miss a 48-yard field goal during overtime, giving the Jets the win. This represents the 5th time this season that the Giants have lost when leading with 2 minutes or less in the 4th quarter, with 3 of them coming at home.\",\"question\":\"Which team scored the only points in overtime?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/ucinlp/drop","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}