{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"66753c25-ad4b-5b86-8f17-011b68785d6e","task_key":"default--validation--35271","task_revision_id":"1","upstream_id":"35271","short_description":"HellaSwag validation 35271","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to house train a puppy [title] Understand how your puppy views the environment. [step] Puppies don't have an inherent understanding of what humans consider right and wrong. They can learn many behaviors, though.\",\"endings\":\"['Playing with your puppy, spending lots of time with it, and handling it require careful eating skills. Here are some of the most important behaviors your puppy can learn : [substeps] Follow your gut.', 'Some basic behaviors for puppies to learn include : [substeps] Independence. Puppies will age faster if they can stand and walk two times every day.', 'A puppy doesn\\\\'t understand that it is \\\" bad \\\" behavior to urinate on your carpet. To your puppy, the carpet is an acceptable surface, just like the grass outside.', 'This includes breaking rules, being mewned over, biting, barking, plucking, pulling, hunting, or playing with another puppy. Instead, your puppy should know the nature of the situation.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}