{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b7e299fd-31a0-566c-a4d7-36288ae112b5","task_key":"default--validation--1979","task_revision_id":"1","upstream_id":"1979","short_description":"HellaSwag validation 1979","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to plan an eco friendly honeymoon [title] Discuss the experience that you would like to have with your fiancé. [step] Ecotourism can range from staying in green lodges to camping to volunteering for a service project. Decide what activities you would like to do, including sightseeing, hiking, train travel, bike travel, local farm visits, snorkeling, swimming and more.\",\"endings\":\"['[title] Invite your fiancé to a group activities. [step] There are a multitude of activities that you could do, including : [substeps] See stars and look at local attractions, including museums, coffee shops, nature preserves, and beaches.', '[substeps] Most eco-friendly honeymoons embrace the natural environment. Ecolodges and resorts are often near wildlife preserves or in places where you do not need to travel by car, boat or plane to see the sights.', 'Plan at least one evening where you can make or use cool-down time with friends, family, or acquaintances. [substeps] It may be helpful to arrange for both you and your fiancé to go camping for the evening.', '[substeps] Review the lifestyle of your prospective partner before booking a trip with her or him. Look into any limitations such as boarding on trains, staying in hotels etc.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}