{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e5076fd5-1d47-5cc9-9f5c-a05cbae9fcca","task_key":"default--validation--18560","task_revision_id":"1","upstream_id":"18560","short_description":"HellaSwag validation 18560","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to train a blind dog [title] Exercise your dog as usual. [step] You should continue to take your dog on walks to keep him healthy and maintain your bond. When you walk your dog, watch for obstacles that may cause your dog to trip and gently redirect your dog as needed.\",\"endings\":\"['[substeps] Using a back or front clip harness rather than a leash may give you a little more control over your dog and make it easier for you to guide him around obstacles. Also, keep in mind that your dog will need reassurance if something noisy startles him.', 'Your dog will naturally ignore these obstacles and will calm down as soon as a sign of danger appears. [substeps] If your dog is simply uninterested, try a walking exercise that can be repeated several times a day.', \\\"Try to calm your dog as much as possible, scolding it if it makes the mistake or waves at you when you're not close enough to keep walking. [substeps] You can also try gently petting your dog or holding its hand or cuddle the dog at the beginning of a walk if you have experience doing so.\\\", '[substeps] Dragging your dog when not in the right direction can lead him to run low on his leash. If your dog has a hard time maintaining a proper pace, you may want to stop (or move) the dog so you stop just long enough to let your dog rest and recover from his injuries.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}