{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"drop","formal_name":"DROP","introduction":"DROP is a reading-comprehension benchmark whose questions require discrete operations over the paragraph — addition, subtraction, comparison, counting, sorting. It is built so that extracting a span cannot answer them.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/ucinlp/drop","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4643ddf7-4ec5-596c-81eb-afb2c22e06a1","task_key":"default--validation--976388a0~2d6060~2d4d99~2d9d7e~2db3b729edffc3","task_revision_id":"1","upstream_id":"976388a0-6060-4d99-9d7e-b3b729edffc3","short_description":"Which team did Puhl sign with last, New York Mets or Kansas City Royals?","config":"default","split":"validation","body":"{\"passage\":\"Puhl was granted free agent status in 1990, and signed with the New York Mets on December 13, 1990. He was released by the Mets during spring training on April 1, 1991, and subsequently signed by the Kansas City Royals on April 25, 1991. His final MLB game was on May 29, 1991, then was released from the Royals on June 9, 1991, ending his MLB career. His career statistics included a .280 batting average, 62 home runs, 676 runs, 435 RBIs, and 217 stolen bases. He accumulated 1,361 hits in 1,531 games. He was particularly effective in postseason play, batting .372 in 13 games over three series.\",\"question\":\"Which team did Puhl sign with last, New York Mets or Kansas City Royals?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/ucinlp/drop","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}