{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a82dde63-dd92-5746-a8e1-aa3dcf46de62","task_key":"default--validation--47512--variant-c42b88c37c96","task_revision_id":"1","upstream_id":"47512--variant-c42b88c37c96","short_description":"HellaSwag validation 47512","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to put nail polish on your dog's nails [title] Make sure the dog is trained. [step] He won't need to know any fancy commands, but he will need to know how to sit and stay for several minutes. [title] Get the dog used to grooming.\",\"endings\":\"[\\\"[step] You should never paint your dog's nails if he isn't accustomed to being groomed, particularly to having his. You need to get him comfortable with the idea of you handling his paws before you start painting his nails.\\\", '[step] Start brushing with the soft tail of a medium sized dog (approximately 4 to 6 weeks old). Ask for attention and take your hands gently while massaging his tail.', '[step] A conditioned dog needs lots of grooming, so get out some now. Tell him to sit down with his tail up and spread his coat to the side.', '[step] Nail polish remover is designed for dogs of all ages, so your dog will need to learn to love using it. One good place to start is to purchase a paw brush that has micro bristles and then make the brush wet.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}