{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"317319ae-7bd3-5083-818b-95fe27a6a445","task_key":"default--validation--32582","task_revision_id":"1","upstream_id":"32582","short_description":"HellaSwag validation 32582","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to street dance [title] Choose the right footwear. [step] You will want sturdy shoes for street dancing. [substeps] Well made sneakers or high top hip hop shoes will work very well for street dancing.\",\"endings\":\"['You want to choose a comfortable size for the job. [title] Talk to the front desk at the green street dancing studio.', \\\"Try to avoid wearing old, beat up shoes. The style of sneakers you wear doesn't matter as much, but it is important that they are sturdy and comfortable.\\\", 'Heels and stilettos are not the same thing. Experts do not recommend these shoes because of the high odds of getting injured in street dancing.', 'Try shoes that are narrow with no sole. The space between your toes should not cause the soles of your feet to wobble.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}