{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"80153419-1e56-5cd5-93c3-03aa0e540951","task_key":"default--validation--7315","task_revision_id":"1","upstream_id":"7315","short_description":"HellaSwag validation 7315","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to find your cat's preferred play style [title] Move toys along the ground if your cat enjoys ground play. [step] Some cats prefer hunting on the ground and as a result enjoy playing with ground toys. This may be because the cat feels it can remain hidden while playing on the ground and stalking toys.\",\"endings\":\"['Alternatively, it could be due to physical restraints. An overweight cat will not be able to jump through the air chasing after toys.', 'However, it may also be because your cat is searching or waits until its prey is out in the open. [substeps] Making sure you remain camouflaged is important when you choose to move toys along the ground.', 'If your cat enjoys ground play, move toys along the ground so they do not distract the cat. [substeps] Keep in mind that cats generally prefer to play using play toys along the ground when they are hunting.', 'This type of play may also be used in outdoor hunting, which a cat may prefer during the day. A result of this type of play is that the cat may hunt on top of ground toys and have fun chasing small birds.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}