{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d7f9a468-7ff4-53af-a345-37adc2f150a7","task_key":"default--validation--22652","task_revision_id":"1","upstream_id":"22652","short_description":"HellaSwag validation 22652","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to tame a feral kitten [title] Put the kitten (s) in a carrier or small room away from other animals. [title] Make sure the kitten has food, water, cat litter, and blankets. [step] Kittens will usual use the litter box.\",\"endings\":\"[\\\"If they go to the bathroom anywhere other than the litter, move the waste into the litter box to show them where they should go. [title] Don't touch the new kitten for at least 24 hours.\\\", '[title] Position the cat so it is in a manner that does not restrict its movement, such as on your lap. [step] You could also provide a place in the room the cat usually searches.', 'However, if they cannot find a suitable place to hide and stay most likely they will begin hunting around the house to find something. [title] Keep the kitten outdoors, preferably away from children and pets.', 'Make sure the kitten sleeps on these materials. Cats like to sleep on soft materials, which make it easier for them to get into and out of their litter box.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}