{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"drop","formal_name":"DROP","introduction":"DROP is a reading-comprehension benchmark whose questions require discrete operations over the paragraph — addition, subtraction, comparison, counting, sorting. It is built so that extracting a span cannot answer them.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/ucinlp/drop","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"768e843c-2c39-5caf-9fcc-e80c8ba4b873","task_key":"default--validation--bbf7a8c5~2d0825~2d4541~2d84f0~2d41f5d494f23a","task_revision_id":"1","upstream_id":"bbf7a8c5-0825-4541-84f0-41f5d494f23a","short_description":"How many points were scored in the first quarter?","config":"default","split":"validation","body":"{\"passage\":\" The Bears and Panthers squared off and it was defensive. The lone score of the first quarter came when Brandon Lloyd recovered a blocked punt and took it nine yards to score. Each team was able to score a field goal as Robbie Gould and John Kasay hit field goals from 26 and 37 yards, respectively. Jason McKie scored a touchdown for the Bears to open the second half, and John Kasay nailed a 45-yard field goal. Jonathan Stewart was able to get into the endzone for his first career touchdown. In the fourth quarter, another Jonathan Stewart run gave the Carolina Panthers the lead and eventually, the victory. Also, WR Muhsin Muhammad (5 receptions for 59 yards) became the 29th player in NFL history to amass 10,000 career receiving yards.\",\"question\":\"How many points were scored in the first quarter?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/ucinlp/drop","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}