{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"44b36bb4-9545-5cba-8450-7a4fe20b08e9","task_key":"default--validation--32571","task_revision_id":"1","upstream_id":"32571","short_description":"HellaSwag validation 32571","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to resist a begging dog [title] Look away from those begging eyes. [step] Giving your dog any attention when it's begging only encourages it to beg more. Plus, if you look at your dog, you're more likely to give in.\",\"endings\":\"[\\\"Look at your own eyes while they're begging, and that will be a signal that it's more likely for him to stop. [substeps] Instead, look at the dog's mouth, and see how drooling is making your dog feel.\\\", \\\"Try looking elsewhere, or even behind the furniture. Your dog will reach for you because you'll be facing it.\\\", \\\"If you give into your dog and feed it food sometimes, that teaches it will be rewarded for this behavior. By ignoring the dog altogether, your dog will learn that begging doesn't get it anywhere.\\\", \\\"Forcing the dog to look at you can hurt your dog's own sense of self-worth. [substeps] Avoid looking at your dog through its eyes.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}