{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a9a3fb60-9339-5c1c-82e7-06802f633f37","task_key":"default--validation--44966","task_revision_id":"1","upstream_id":"44966","short_description":"HellaSwag validation 44966","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to take your pet bird outside [title] Buy a harness. [step] The aviator harness is recommended by avian vets for taking birds outside. It will take some time to harness train your bird, so don't rush it or there could be permanent effects.\",\"endings\":\"['Always follow the harness instructions (the aviator comes with a dvd guide). The aviator comes in sizes for all birds and has many innovative features to keep your bird safe.', '[title] Cut the harness to the dimensions appropriate. [step] The aviator harness needs to be same in length as your bird to make it easier to remove.', \\\"[substeps] The aviator harness can be a tool that can improve your bird's skin and body. It will be more expensive, however, than a rope harness.\\\", 'To harness train your bird, squeeze some egg yolk into a plastic or rubber tube. Hold on tightly as your bird tries to free itself from the tube.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}