{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e673a0f7-098f-54cb-9fa1-f3eed1ec4c2e","task_key":"default--validation--23794","task_revision_id":"1","upstream_id":"23794","short_description":"HellaSwag validation 23794","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to discipline your puppy [title] Start early and be consistent. [step] You should start disciplining your puppy as soon as you bring him home. Puppies learn quickly and should be shown what behavior is unacceptable in your house.\",\"endings\":\"[\\\"However, it's important to start letting him know that going out, eating, and putting away toys or food on a daily basis, is unacceptable. [substeps] Consistency enables your puppy to learn how to obey his owner.\\\", 'If you ignore commands, it can be a serious problem up close and personal for them. Your puppy should be trained and encouraged to use inappropriate commands such as \\\" sit \\\" or \\\" stay \\\" in order to avoid being punished.', 'This includes cursing, mean language, loud slurs, or even yelling, though these behaviors are not acceptable unless the puppy is leash trained. Correct behavior is never tolerated, no matter what.', 'This also means that you must be consistent when responding to bad behavior. If not, your puppy will be confused, making it harder to prevent the bad behavior.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}