{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"drop","formal_name":"DROP","introduction":"DROP is a reading-comprehension benchmark whose questions require discrete operations over the paragraph — addition, subtraction, comparison, counting, sorting. It is built so that extracting a span cannot answer them.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/ucinlp/drop","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"12cd13f1-d0e0-50a2-806e-73aea517929f","task_key":"default--validation--3b8aae6c~2d2afb~2d43a5~2da045~2d1cabd364c3de","task_revision_id":"1","upstream_id":"3b8aae6c-2afb-43a5-a045-1cabd364c3de","short_description":"How many touchdowns were scored in the game?","config":"default","split":"validation","body":"{\"passage\":\" Coming off their home win over the Vikings, the Saints flew to Candlestick Park for a Week 2 Monday Night duel with the San Francisco 49ers.  New Orleans would deliver the opening punch in the first quarter as 49ers center David Baas' high snap deep within San Francisco territory went out of the back of the endzone for a safety.  The Saints would add onto their lead as quarterback Drew Brees hooked up with running back Reggie Bush on a 6-yard touchdown pass.  The 49ers would answer in the second quarter as quarterback Alex Smith found running back Frank Gore on a 12-yard touchdown pass. San Francisco would take the lead with running back Anthony Dixon's 2-yard touchdown run, yet New Orleans would respond with Brees' 3-yard touchdown pass to tight end David Thomas.  The Saints would add onto their lead in the fourth quarter as kicker Garrett Hartley made a 46-yard and a 19-yard field goal.  The 49ers would strike back as Gore got a 7-yard touchdown run, followed by Smith's 2-point conversion pass to tight end Vernon Davis.  In the end, New Orleans got the last laugh as Hartley booted the game-ending 37-yard field goal.\",\"question\":\"How many touchdowns were scored in the game?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/ucinlp/drop","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}