{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"drop","formal_name":"DROP","introduction":"DROP is a reading-comprehension benchmark whose questions require discrete operations over the paragraph — addition, subtraction, comparison, counting, sorting. It is built so that extracting a span cannot answer them.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/ucinlp/drop","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ca74b5ae-822c-5ad8-aa24-7d7653695c49","task_key":"default--validation--03190fb4~2d66f0~2d4688~2d89f7~2dc3f2351c87f4","task_revision_id":"1","upstream_id":"03190fb4-66f0-4688-89f7-c3f2351c87f4","short_description":"How many years did the great expeditions last?","config":"default","split":"validation","body":"{\"passage\":\"The British Admiralty had decided to prosecute the Ear Jenkins war against the Spanish settlements, though on a different scale from that of the great expeditions of 1741 and 1742. Sir Chaloner Ogle, who had replaced Admiral Edward Vernon after the defeat at the Battle of Cartagena de Indias in 1741, prepared an invasion of another important commercial port on the Spanish Main. Believing La Guaira to be not well defended, Sir Chaloner Ogle wanted to take advantage and attack. On 22 February 1743, Sir Charles Knowles sailed from the island of Antigua with 19 ships:\",\"question\":\"How many years did the great expeditions last?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/ucinlp/drop","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}