{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9ccb5bc6-bc6b-5db8-9621-570ddee7b63f","task_key":"default--validation--11799","task_revision_id":"1","upstream_id":"11799","short_description":"HellaSwag validation 11799","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to keep a cat calm during a move [title] Make a mess. [step] Take out the carrier, the moving boxes, the sticky tape, the bubble wrap, everything that will be needed in the week of the move. The cat has to get used to these things on its own and make sure they're not intimidating.\",\"endings\":\"['Put things, plants, clothes, bedding etc, in the carrier. [substeps] If you can, try feeding your cat a hearty enchilada before leaving the home.', \\\"[title] Get it pre-assembled so it'll be easy to unpack. [step] Ensure that the cat can move at its own pace.\\\", '[title] Leave everything on the floor and take your cat outside. [step] Throw in any hissing or growling, anything you can think of that might scare the cat.', '[title] Walk around with the cat and introduce it to the new place. [step] The more it knows, the less stressful moving with the cat will be.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}