{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"decfa308-ed72-5aaf-9c14-89c3d5d70d48","task_key":"default--validation--12904","task_revision_id":"1","upstream_id":"12904","short_description":"HellaSwag validation 12904","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to help a fearful kitten feel more confident [title] Introduce your kitten to new things slowly. [step] In order to get your cat used to new things, you should introduce the kitten to things slowly. Let your kitten smell a new thing and approach it at their own pace.\",\"endings\":\"['This may require some practice for a while, but the longer the kitten gets to become familiar with new behaviors, the better your chance of helping your kitten overcome an anxiety attack. [substeps] Handle the kitten for 20 minutes a day.', '[substeps] For example, if your kitten seems afraid of pots and pans clanging around in the kitchen, show the kitten a pan and let the cat smell the item. Then show the kitten the clanging sound that the pan makes so it will understand the sound and get used to it over time.', 'To be sure your kitten loves the interaction, encourage it to interact with new sights, sounds, and smells. You could also try introducing your kitten to new places, such as where you used to take your kitten on trips.', 'Place your arm around its waist or up near its ears. Pet your kitten and let it smell a new thing.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}