{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"drop","formal_name":"DROP","introduction":"DROP is a reading-comprehension benchmark whose questions require discrete operations over the paragraph — addition, subtraction, comparison, counting, sorting. It is built so that extracting a span cannot answer them.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/ucinlp/drop","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2092685e-b120-5852-a8e2-7dafe0ad6e24","task_key":"default--validation--2150c2d4~2d6a86~2d405b~2dbf9c~2da103aa212a6a","task_revision_id":"1","upstream_id":"2150c2d4-6a86-405b-bf9c-a103aa212a6a","short_description":"Which year did the union have more members 1980 or 1970?","config":"default","split":"validation","body":"{\"passage\":\"By 2011 fewer than 7% of employees in the private sector belonged to unions. The UAW's numbers of automobile union members are representative of the manufacturing sector: 1,619,000 active members in 1970, 1,446,000 in 1980, 952,000 in 1990, 623,000 in 2004, and 377,000 in 2010 . By 2014, coal mining had largely shifted to open-pit mines in Wyoming, and there were only 60,000 active coal miners.  The UMW has 35,000 members, of whom 20,000 were coal miners, chiefly in underground mines in Kentucky and West Virginia. By contrast it had 800,000 members in the late 1930s. However it remains responsible for pensions and medical benefits for 40,000 retired miners, and for 50,000 spouses and dependents.\",\"question\":\"Which year did the union have more members 1980 or 1970?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/ucinlp/drop","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}