{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"drop","formal_name":"DROP","introduction":"DROP is a reading-comprehension benchmark whose questions require discrete operations over the paragraph — addition, subtraction, comparison, counting, sorting. It is built so that extracting a span cannot answer them.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/ucinlp/drop","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e60a30ed-7ac1-5f45-a3ab-adeb54d0ec1b","task_key":"default--validation--270d749a~2dd574~2d4469~2db76d~2d39a384d24edd","task_revision_id":"1","upstream_id":"270d749a-d574-4469-b76d-39a384d24edd","short_description":"Which quarterback threw for more yards, Moore or Manning?","config":"default","split":"validation","body":"{\"passage\":\" The Giants again found themselves trailing the winless Dolphins entering the fourth quarter. New York was down 17-10 behind two rushing touchdowns, one by wildcat quarterback Steve Slaton and the other by starter Matt Moore, and a Dan Carpenter field goal. Lawrence Tynes recorded a field goal of his own while Eli Manning threw a touchdown to Mario Manningham in the second quarter. After Tynes recorded his second field goal in the fourth, Manning found Victor Cruz for a 25-yard touchdown with 2:30 left to give the Giants the victory. Cruz recorded 99 yards on seven receptions while Manning threw for 349 yards and two touchdowns. Reggie Bush led all runners with 103 yards and Moore threw for 138 and an interception.\",\"question\":\"Which quarterback threw for more yards, Moore or Manning?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/ucinlp/drop","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}