{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"53d93624-a9d6-5c7a-80e4-bf7e7fe140bf","task_key":"default--validation--12764","task_revision_id":"1","upstream_id":"12764","short_description":"HellaSwag validation 12764","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to introduce a new puppy [title] Identify what the puppy will need and who will provide these needs. [step] Puppies require a lot of care, love, and patience. If you are bringing a new puppy into your home, you should take some time to identify what its needs will be and decide who will be responsible for each of these needs.\",\"endings\":\"['[substeps] Puppies under six weeks are often responsible for between two and four days of daily care. Puppies of eight weeks and under two weeks require daycare, including attention.', 'Let the puppy know that you look after it and feel safe helping it meet your expectations. This may help to reassure the puppy that you will provide it with everything it needs.', 'Caring for a dog is a great way to teach children responsibility and compassion for animals. Try to get the whole family involved in caring for your new puppy.', '[substeps] Dogs are very vulnerable to physical and genetic injuries. A household dog may not have the appropriate amount of time to play with the dog, exercise, and socialize the puppy.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}