{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"drop","formal_name":"DROP","introduction":"DROP is a reading-comprehension benchmark whose questions require discrete operations over the paragraph — addition, subtraction, comparison, counting, sorting. It is built so that extracting a span cannot answer them.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/ucinlp/drop","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c38fb86b-944d-5cee-b06d-3ce149d5cd6c","task_key":"default--validation--58706586~2d1d76~2d41fe~2db39d~2d7de89f80f8ce","task_revision_id":"1","upstream_id":"58706586-1d76-41fe-b39d-7de89f80f8ce","short_description":"How many touchdown passes did Jay Cutler throw?","config":"default","split":"validation","body":"{\"passage\":\" The Bears kicked off their season at home with an NFC North duel against the Detroit Lions. In the first quarter, the Bears took the early lead with kicker Robbie Gould nailing a 20-yard field goal. The Lions responded with 2 TD runs (7 and 4 yards) by running back Jahvid Best in the second quarter. The Bears fought back with quarterback Jay Cutler completing an 89-yard screen pass to running back Matt Forté for a TD, followed by Gould's 31-yard field goal after a Stafford fumble. The Bears increased their lead in the fourth quarter when Cutler threw a 28-yard TD pass to Matt Forté. The Bears tried to extend their lead to 7, but failed on their 2-point conversion. The game ended in somewhat controversial fashion, however, as an apparent touchdown reception in the final minute by Detroit receiver Calvin Johnson was overturned via a little known rule (completing the process of a catch) and ruled incomplete.\",\"question\":\"How many touchdown passes did Jay Cutler throw?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/ucinlp/drop","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}