{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e8ea13ae-e5e5-5c69-8720-9cde2f78e734","task_key":"default--validation--25038","task_revision_id":"1","upstream_id":"25038","short_description":"HellaSwag validation 25038","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to tame a cockatiel [title] Stay quiet and unobtrusive around a new cockatiel. [step] Do not attempt begin taming until it has spent a few weeks getting accustomed to its new environment. Keep the cockatiel in a quiet, calm area.\",\"endings\":\"['Do not talk or tap its head until it associates your voice with the sounds of approaching thunder, quail, or black cats. These noises will scare and confuse the cockatiel.', 'Under the same roof, away from the kitchen and other noise, try to \\\" talk \\\" it down. You can even groom or bathe it as a treat to get it to calm down.', 'Keep a squeaky cage near an open window or near other birds so that the cockatiel can hear you without fear. You could also put it with a newspaper or folded blanket.', '[title] Talk to the cockatiel from outside the cage. [step] You may say anything you like, as long as your voice stays calm, without sudden changes in volume.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}