{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"drop","formal_name":"DROP","introduction":"DROP is a reading-comprehension benchmark whose questions require discrete operations over the paragraph — addition, subtraction, comparison, counting, sorting. It is built so that extracting a span cannot answer them.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/ucinlp/drop","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"18ad44cd-71a1-5d28-8aaf-1d53ca422697","task_key":"default--validation--6537c462~2d1478~2d4615~2d8ad1~2d3dc86bc1a373","task_revision_id":"1","upstream_id":"6537c462-1478-4615-8ad1-3dc86bc1a373","short_description":"How many divisions entered the deserted city of Pyongyang?","config":"default","split":"validation","body":"{\"passage\":\"The First Division under Konishi Yukinaga proceeded northward, and sacked Pyongsan, Sohung, Pungsan, Hwangju, and Chunghwa along the way. At Chunghwa, the Third Division under Kuroda Nagamasa joined the First, and continued to the city of Pyongyang located behind the Taedong River. A total of 10,000 Korean troops guarded the city against the 30,000 advancing Japanese. They were led by various commanders including the Generals Yi Il and Kim Myong-won, and their defense preparations had assured that no boats were available for Japanese use. At night, the Koreans silently crossed the river and launched a successful surprise attack against the Japanese encampment. However, this alerted the rest of the Japanese army, which attacked the rear of the Korean positions and destroyed the remaining reinforcements crossing the river. The remainder of the Korean troops then retreated back to Pyongyang, and the Japanese troops gave up their pursuit of the Koreans to observe the manner in which the Koreans had crossed the river. The next day, using what they had learned from observing the retreating Korean troops, the Japanese began sending troops to the other shore over the shallow points in the river, in a systematic manner, and at this sight the Koreans abandoned the city overnight. On July 20, 1592, the First and Third Divisions entered the deserted city of Pyongyang. In the city, they managed to capture 100,000 tons of military supplies and grain.\",\"question\":\"How many divisions entered the deserted city of Pyongyang?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/ucinlp/drop","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}