{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"drop","formal_name":"DROP","introduction":"DROP is a reading-comprehension benchmark whose questions require discrete operations over the paragraph — addition, subtraction, comparison, counting, sorting. It is built so that extracting a span cannot answer them.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/ucinlp/drop","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"660ab9cd-39b2-5bf5-a298-dec74582e144","task_key":"default--validation--da9b7bde~2daac5~2d45c5~2d83ea~2d6f7587a3062d","task_revision_id":"1","upstream_id":"da9b7bde-aac5-45c5-83ea-6f7587a3062d","short_description":"How many points did the Packers score?","config":"default","split":"validation","body":"{\"passage\":\" For the first time in franchise history, the Packers lost to the Ravens at home. It was the second time in 11 years that the team was shut-out at home. After going the previous two games without a turnover, the Packers committed five: two fumbles and three interceptions. Brett Hundley threw three interceptions, including one in the end-zone on the team's opening drive, bringing his season interception total to seven, matching Rodgers' total from the entire 2016 regular season. Packers' rookie running back DeVante Mays fumbled on the first two carries of his career, losing one. Field position was key as the Ravens' special teams often kept the Packers starting drives deep in their own territory.\",\"question\":\"How many points did the Packers score?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/ucinlp/drop","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}