{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"drop","formal_name":"DROP","introduction":"DROP is a reading-comprehension benchmark whose questions require discrete operations over the paragraph — addition, subtraction, comparison, counting, sorting. It is built so that extracting a span cannot answer them.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/ucinlp/drop","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"198f7b80-fe56-520c-845d-b27bbe3640db","task_key":"default--validation--9fa653c5~2d4c17~2d492b~2d93c5~2d5738184e306c","task_revision_id":"1","upstream_id":"9fa653c5-4c17-492b-93c5-5738184e306c","short_description":"How many catches did Franks have?","config":"default","split":"validation","body":"{\"passage\":\" The Giants played their Week 2 home opener against the Green Bay Packers.  After both teams missed a field goal to open the game, Green Bay got on the board first with a 6-yard run by rookie DeShawn Wynn. The Giants responded with a 26-yard scoring strike by Eli Manning to Plaxico Burress. The Giants got a Lawrence Tynes field goal and a 10-7 half time lead. In the second half, the Packers drove 51 yards to start the second half. Favre capped off the scoring drive with a 2-yard pass to Bubba Franks for a 14-10 lead the Packers would not relinquish. The Giants responded with another drive but once again, an unsportsmanlike conduct penalty, this one on Amani Toomer, hurt the Giants. Tynes kicked a second field goal, pulling the Giants to within one. With the loss, the Giants would fall to 0-2 for the first time since 1996.\",\"question\":\"How many catches did Franks have?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/ucinlp/drop","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}