{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"drop","formal_name":"DROP","introduction":"DROP is a reading-comprehension benchmark whose questions require discrete operations over the paragraph — addition, subtraction, comparison, counting, sorting. It is built so that extracting a span cannot answer them.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/ucinlp/drop","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b82529eb-2ff3-5230-818e-73787556722b","task_key":"default--validation--c3061d02~2d9549~2d4398~2d97bd~2d9d16cbfb8246","task_revision_id":"1","upstream_id":"c3061d02-9549-4398-97bd-9d16cbfb8246","short_description":"How many times was a new theme introduced in the month of January?","config":"default","split":"validation","body":"{\"passage\":\"The themes and bumper music that were used from January 22, 1979 to January 1, 1988 were composed by Edd Kalehoff, also known for themes on other television series and game shows. The longest-running theme, also composed by Kalehoff, was used from January 4, 1988 to November 15, 2002. Along with an update to the graphics and presentation, the theme was updated on November 18, 2002 and was used until January 1, 2010. A new theme, logo, and set for the shows Miami headquarters debuted on January 4, 2010. This theme lasted until April 27, 2012. On April 30, 2012, a new virtual theme was introduced to the show with new graphics and music. On March 4, 2013, another new set of graphics was introduced, yet the theme music remained the same.  On January 4, 2016, the graphics were changed again; this time they are modeled after CNBCs current graphics package (which itself has been used since October 2014).\",\"question\":\"How many times was a new theme introduced in the month of January?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/ucinlp/drop","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}