{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"drop","formal_name":"DROP","introduction":"DROP is a reading-comprehension benchmark whose questions require discrete operations over the paragraph — addition, subtraction, comparison, counting, sorting. It is built so that extracting a span cannot answer them.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/ucinlp/drop","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ec20d85e-ff75-5d55-b415-6466a9345a78","task_key":"default--validation--5fc913ff~2dbe3f~2d491b~2da026~2dfb71a63e3483","task_revision_id":"1","upstream_id":"5fc913ff-be3f-491b-a026-fb71a63e3483","short_description":"Which Cardinal died first from the plaque of France, Foresta or Farinier?","config":"default","split":"validation","body":"{\"passage\":\"The reason for his retirement to Auriol is not far to seek.  The plague was raging in southern France again in 1361 and 1362.   Cardinal Pierre des Près died on 16 May 1361; Cardinal Petrus de Foresta, died on 7 June 1361; Cardinal Guillaume Farinier, died on 17 June 1361; Cardinal Guillaume Court, O.Cist., died on  12 June 1361; Cardinal Petrus Bertrandi, died on 13 July 1361; Cardinal Jean de Caraman, died on 1 August 1361; Cardinal Bernard de la Tour, died on 7 August 1361; Cardinal Francesco degli Atti, died on 25 August 1361; and Cardinal Pierre de Cros died in September 1361.  In addition it was estimated that some 6000 persons, and more than 100 bishops died in 1361.  Cardinal Nicolas Roselli (1357-1362) of Tarragona died at Majorca on 28 March 1362, though not of the plague.\",\"question\":\"Which Cardinal died first from the plaque of France, Foresta or Farinier?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/ucinlp/drop","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}