{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7c08133a-c186-551a-affe-3b7e835912da","task_key":"default--validation--1120","task_revision_id":"1","upstream_id":"1120","short_description":"HellaSwag validation 1120","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to prepare your dog for a road trip [title] Visit your veterinarian. [step] Perhaps the first step in preparing your dog for a road trip is to make sure that he or she is healthy enough to take the trip. This should be no problem if you regularly take your dog to the vet for checkups and other issues.\",\"endings\":\"['If you have not : [substeps] Make an appointment with the vet. Talk to your vet to make sure your dog is up to date on important vaccinations.', 'Your veterinarian will be able to provide you with vaccinations, in addition to taking your dog in for professional vet visits. [substeps] Although canine infection is rare, your veterinarian may be able to provide vaccine for canine infectious diseases, like measles.', 'Mention that your dog needs to follow a certain food regimen and behaviors during the trip. [substeps] Do not try to \\\" test \\\" your dog\\\\'s blood for any diseases.', \\\"Make sure your veterinarian shows that your dog is up to date on his or her vaccinations. [substeps] Be sure to bring your dog's current medication and acceptable lab coat.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}