{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ad5cd8b6-8e4e-57b1-88ea-31c72a5572fd","task_key":"default--validation--50244","task_revision_id":"1","upstream_id":"50244","short_description":"HellaSwag validation 50244","config":"default","split":"validation","body":"{\"activity_label\":\"Cars & Other Vehicles\",\"ctx\":\"[header] How to buy a classic volvo [title] Know what kind of volvo vehicle you are looking for and what kind of features fit your style. [substeps] Volvos are available in station wagons, sedan models, suvs and compact sedans as well as coupes. Volvo vehicles have high safety standards and generally get good gas mileage.\",\"endings\":\"['The grand prix avr with greater efficiency and the buicks offer high mileage options for your vehicle. Rachel wyen convertible 1x550lds, lincoln 2450 and dc durgels x17 models have high mileage features for the same vehicles as the mini vac.', 'Models and sedacs are also available on bridge stands and service rails. Single engine volvo vehicles have no visible signs on their doors or partition.', 'The oldest volvos came off the assembly line in 1927, and the model is referred to as the volvo � v 4. Because volvo group was one of the largest commercial vehicle manufacturers in the world before it was bought out in 1998 by ford, volvos can be found all over the world.', 'They are sometimes called mk6 for more customizable cars, especially a honda reny 2005. Many volvo are shade colored to enrich their style.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}