{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"dff26dc0-3345-55a0-978f-649191e5577e","task_key":"default--validation--34019","task_revision_id":"1","upstream_id":"34019","short_description":"HellaSwag validation 34019","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to be safe around stray dogs [title] Differentiate between stray animals and lost pets. [step] If you happen upon a dog, you will most likely be able to tell whether the dog is a stray or a lost pet. [substeps] Stray dogs may not have had any positive human contact and are not socialized towards humans.\",\"endings\":\"['These pets can sneak under your rug and get caught. Lost pet owners often opt to take the dog out with them and make sure they socialize the dog with the care and safety of a stray puppy.', 'They will drop the dog off, get lost and walk away following the dog for a short period of time. If the dog seems to be acting indifferent around you, this may be a dog that wants to protect itself.', 'They tend to be wary of humans and their fear can be triggered into aggression. Pets that are lost are more likely to approach you and be comfortable around humans.', 'For these dogs to be considered stray, they must also have lost all of their dignity and respect, and were mistreated. Lost pets also may not require reporting the dog to the police.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}