{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7e66a18b-dbbb-5589-9c1b-75df6404644e","task_key":"default--validation--29288","task_revision_id":"1","upstream_id":"29288","short_description":"HellaSwag validation 29288","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to find homes for a litter of kittens [title] Get your kittens vaccinated. [step] It will be harder to find a good home for your litter of kittens if they are not vaccinated. Take your litter of kittens to the vet when they are about eight weeks old so they can begin receiving a series of routine vaccinations.\",\"endings\":\"['[title] Buy cat beds for your neighborhood that can fit your litter mates. [step] Plus, make sure you have good substrate as kittens will chew through more carpet than magazines and newspapers.', \\\"[title] Calculate your cat's calorie needs. [step] Kittens aged between four and nine weeks old get most of their calorie from their food.\\\", 'Your vet will advise you regarding which vaccinations your kitten needs and when. [substeps] When taking the kittens to the vet, have them dewormed and treated for any other maladies-such as parasites-that might be present.', 'Most health care centers and veterinary hospitals offer commercial package vaccines. [substeps] Visit the vet regularly and ask them to give you a unique package that includes the proper-name cat (sp) for your location.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}