{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"drop","formal_name":"DROP","introduction":"DROP is a reading-comprehension benchmark whose questions require discrete operations over the paragraph — addition, subtraction, comparison, counting, sorting. It is built so that extracting a span cannot answer them.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/ucinlp/drop","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"86ae568d-9429-543c-a6c5-44e680422279","task_key":"default--validation--9f886606~2d9850~2d42f9~2dae6b~2d147eac558517","task_revision_id":"1","upstream_id":"9f886606-9850-42f9-ae6b-147eac558517","short_description":"How many games had the Patriots lost all season?","config":"default","split":"validation","body":"{\"passage\":\" Meeting for the first time since Super Bowl XLVI, the Giants host the undefeated Patriots. Commissioner Roger Goodell was in attendance. The Giants looked to stop the streaking Patriots in a game where both teams saw touchdowns wiped out on penalties. The Giants trailed by 2 at the two-minute warning, and on second down Eli Manning tossed a pass in the end zone to Odell Beckham Jr.. Beckham appeared to have the catch, but as he brought the ball up in celebration, it was knocked out by Malcolm Butler in a play eerily similar to a drop by Lee Evans in the closing seconds of the 2011 AFC Championship Game. Manning scrambled for a loss on third down, and Josh Brown kicked the go-ahead field goal to give New York a lead. However, a last second game-winning field goal by the Patriots kicker Stephen Gostkowski ended the Giants hopes. Although he was cleared to play, Victor Cruz, a day later, announced he would not play this season as he underwent surgery to repair an injured left calf.\",\"question\":\"How many games had the Patriots lost all season?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/ucinlp/drop","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}