{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"dc9d417b-9361-58a8-9846-f07e01c1da4e","task_key":"default--validation--25364","task_revision_id":"1","upstream_id":"25364","short_description":"HellaSwag validation 25364","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to throw a couple's wedding shower [title] Do a beach or pool theme. [step] If the bride and groom like to be outside in the sun, go for a beach or pool theme. If you have access to a pool, plan a pool theme with poolside games and activities.\",\"endings\":\"['You can have fun outdoors in the sun, meeting up around the lake, and other activities. [substeps] If you have access to a hot tub or spa, go for a clean water theme.', \\\"If you don't have access to a hot tub, have guests put their swimsuits in the bathtub and run around in the water. Put some coconut water into the water first and then put some sand into the water.\\\", 'Plan whether you want to have dip-ins, magic showings, walks, tropics, or the eiffel tower. [substeps] There is also a group activity also for planning a shower, so choose the group activity well in advance.', 'You can also turn the pool into a beach complete with fake sand and beach umbrellas. [substeps] You can then have guests bring pool toys, tropical drinks, and their swimsuits for the shower.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}