{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7c4ff9e3-1286-5981-b5eb-0ebeacf6d09f","task_key":"default--validation--13174","task_revision_id":"1","upstream_id":"13174","short_description":"HellaSwag validation 13174","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to get the complete surfer look and lifestyle [title] For clothes, try shopping at anywhere that sells roxy, billabong, o'neill and any other surfer brands. [step] If you like a bit of a preppy surfer look, try shopping at hollister. Don't just wear brands though, mix up your look with clothes from other stores.\",\"endings\":\"['[title] For shoes, go for vans, toms, flip-flops or sandals-leather sandals are best. [title] For your skin, try to use products with natural ingredients, like for example liz earle products.', \\\"You can also try picking out old women's bikinis or board shorts. [title] Look for any, many surfer accessories.\\\", '[title] For footwear, try buying boots, converse, hiking boots, uggs or cowboy boots at amazon. [step] Take your shoes off and wear old pair of cowboy boots, check with outside fabrics stores and start wearing your boots on rainy days.', '[title] For makeup, try a lovely assortment of shades of red, pink and green. [title] If you are a natural surfer, you can get a lot of pale skin too.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}