{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"drop","formal_name":"DROP","introduction":"DROP is a reading-comprehension benchmark whose questions require discrete operations over the paragraph — addition, subtraction, comparison, counting, sorting. It is built so that extracting a span cannot answer them.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/ucinlp/drop","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6a5d929a-8dae-5ebd-9b06-ee7784e0006b","task_key":"default--validation--12d2bbe2~2d263a~2d4e90~2d9035~2dec44a1d8e9d9","task_revision_id":"1","upstream_id":"12d2bbe2-263a-4e90-9035-ec44a1d8e9d9","short_description":"Who won the game?","config":"default","split":"validation","body":"{\"passage\":\" Coming off their divisional home win over the Panthers, the Saints flew to the Edward Jones Dome for a Week 10 duel with the St. Louis Rams. After a scoreless first quarter, New Orleans got the game's inaugural points in the second quarter with running back Reggie Bush getting a 3-yard touchdown run. The Rams would respond with quarterback Marc Bulger completing a 29-yard touchdown pass to wide receiver Donnie Avery. New Orleans would answer with quarterback Drew Brees hooking up with Bush on a 15-yard touchdown pass, yet St. Louis would close out the half with a 2-yard touchdown run from running back Steven Jackson. The Saints would begin the third quarter with wide receiver Courtney Roby returning the second half's opening kickoff 97 yards for a touchdown. The Rams would stay close with kicker Josh Brown nailing a 32-yard field goal. New Orleans would extend their lead in the fourth quarter Brees finding wide receiver Robert Meachem on a 27-yard touchdown pass. St. Louis tried to catch up as Bulger found Avery again on a 19-yard touchdown pass (with a failed 2-point conversion), yet the defense prevented the Rams from getting any closer.\",\"question\":\"Who won the game?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/ucinlp/drop","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}