{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f79186b6-9ae2-5e66-a6e4-52e4578584df","task_key":"default--validation--45498","task_revision_id":"1","upstream_id":"45498","short_description":"HellaSwag validation 45498","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to keep a happy hamster [title] Get to know your hamster. [step] Spend some time watching your hamster and getting to know his personality and mannerisms. All hamsters will have their own personalities, so you should try to get know your hamster's and respond to it accordingly.\",\"endings\":\"['A good example of a positive personality for a hamster is a willingness to bring home new mice. [substeps] Choose a hamster to socialize with you.', 'Some hamsters will be more shy and reclusive, and you should respect this and not disturb him unduly if this is the case. [substeps] For example, your hamster might lower himself to the ground and flatten out when you approach or attempt to pick him up.', 'Start slow and talk to your hamster about his behavior and his likes and dislikes. You can also find online sites like pet stores and pet stores where your hamsters would eat mice or rats.', 'Once you get to know your hamster, you can begin his training. [substeps] Your hamster\\\\'s name is ketoconazole, and he\\\\'s a part of the \\\" bitten \\\" club at your school.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}