{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2cb2fd08-415e-5cb5-99aa-73c275b57701","task_key":"default--validation--48034","task_revision_id":"1","upstream_id":"48034","short_description":"HellaSwag validation 48034","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to train a dog pack to be calm and submissive [title] House your dogs separately, as a temporary measure. [step] Before you even begin training your dogs, you're going to have to adopt them. However, when you adopt them, make sure to house them separately.\",\"endings\":\"[\\\"This way, you'll have one large dog and one small dog. If possible, give your dogs separate rooms, and give your smaller ones a place to retreat to to if they find it difficult to calm down.\\\", \\\"By ensuring that your dogs are kept separate, you'll be helping to train them to be calm and submissive. [substeps] Make sure that your dog is not out of control or frightened when you're house training them.\\\", \\\"This way, you'll be able to train them and incorporate them into your family individually. [substeps] The best way is to adopt one dog at a time.\\\", 'This will have the added benefit of preventing them from killing each other, or potentially ending up cold turkey. [substeps] Make sure your house is safe and good for them.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}