{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"15018b2c-6466-5e9b-9773-45799b73bc38","task_key":"default--validation--28964","task_revision_id":"1","upstream_id":"28964","short_description":"HellaSwag validation 28964","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to take a shower if you don't want to [title] Turn on the water. [step] Sometimes in order to accomplish what feels like a monumental task, you have to start small. For now, just go and turn on the shower.\",\"endings\":\"['If you cannot feel your legs or back, or you cannot stand to the side or down, just turn on the water. Make sure any harsh soaps you use, which can easily dry dry your skin is avoided as much as possible as well.', \\\"Don't step on the faucet until you get dry and put on a face mask before you hit the water. [substeps] You don't have to use the sink to clean your hands when showering.\\\", 'This will give your entire body a \\\" working \\\" opportunity to take stock of this water and the waves you\\\\'ve been getting out of it. [substeps] Any shower begins as a brief, quick warm-up, followed by an at-home bath or bath.', \\\"Get the water nice and warm and think about how good it would feel to climb inside. You'll probably find it much easier to shower once you're actually in the bathroom and have everything all ready to go.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}