{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"drop","formal_name":"DROP","introduction":"DROP is a reading-comprehension benchmark whose questions require discrete operations over the paragraph — addition, subtraction, comparison, counting, sorting. It is built so that extracting a span cannot answer them.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/ucinlp/drop","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"70914225-3f86-5346-8382-659baf9b1905","task_key":"default--validation--90cc7634~2dcc69~2d4075~2da74f~2d958ea0b4a00f","task_revision_id":"1","upstream_id":"90cc7634-cc69-4075-a74f-958ea0b4a00f","short_description":"how many yards did Maroney run?","config":"default","split":"validation","body":"{\"passage\":\" Hoping to preserve their division lead, the Patriots flew to Paul Brown Stadium for a Week 4 showdown with the Cincinnati Bengals.  In the first quarter, the Bengals opened the scoring with two Shayne Graham field goals.  The Patriots would get on the board in the second quarter, as Maroney ran in for an 11-yard touchdown and Brady completed a 25-yard touchdown pass to Gabriel.  Down by eight in the third quarter, running back Rudi Johnson ran two yards for a touchdown, the final Cincinnati points of the game. The Patriots took their one-point lead and turned it into a 25-point win, starting with a Maroney 25-yard touchdown run.  In the fourth quarter, Gostkowski kicked a 24-yard field goal, Dillon scored from one yard out on a run, and Brady completed a three-yard touchdown pass to tight end Daniel Graham.  With their victory, the Patriots advanced to 3-1 and maintained their lead in the AFC East.\",\"question\":\"how many yards did Maroney run?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/ucinlp/drop","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}