{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"drop","formal_name":"DROP","introduction":"DROP is a reading-comprehension benchmark whose questions require discrete operations over the paragraph — addition, subtraction, comparison, counting, sorting. It is built so that extracting a span cannot answer them.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/ucinlp/drop","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"03d2428c-9568-5d79-859d-df219c0772b7","task_key":"default--validation--bf2f4423~2d3e12~2d42fc~2dadf3~2d7fd06547ef70","task_revision_id":"1","upstream_id":"bf2f4423-3e12-42fc-adf3-7fd06547ef70","short_description":"What event happened first, James Craig signing an agreement, or the Ulster…","config":"default","split":"validation","body":"{\"passage\":\"The fact that the Irish Civil War was fought between Irish Nationalist factions meant that the sporadic conflict in Northern Ireland ended. Collins and Sir James Craig signed an agreement to end it on 30 March 1922, but, despite this, Collins covertly supplied arms to the Northern IRA until a week before his death in August 1922. Because of the Irish Civil War, Northern Ireland was able to consolidate its existence and the partition of Ireland was confirmed for the foreseeable future. The continuing war also confirmed the northern Unionists' existing prejudices against the ethos of all shades of nationalism. This might have led to open hostilities between North and South had the Irish Civil War not broken out. Indeed, the Ulster Special Constabulary  that had been established in 1920  was expanded in 1922 rather than being demobilised. In the event, it was only well after their defeat in the Civil War that anti-treaty Irish Republicans seriously considered whether to take armed action against British rule in Northern Ireland . The northern units of the IRA largely supported the Free State side in the Civil War because of Collins's policies, and over 500 of them joined the new Free State's National Army. The cost of the war and the budget deficit it caused was a difficulty for the new Free State and affected the Boundary Commission negotiations of 1925, which were to determine the border with Northern Ireland. The Free State agreed to waive its claim to predominantly Nationalist areas in Northern Ireland and in return its agreed share of the Imperial debt under the 1921 Treaty was not paid.\",\"question\":\"What event happened first, James Craig signing an agreement, or the Ulster Special Constabulary?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/ucinlp/drop","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}