{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fbc0b547-1db5-5406-a6f4-b0459b74052d","task_key":"default--validation--5537","task_revision_id":"1","upstream_id":"5537","short_description":"HellaSwag validation 5537","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to understand your rabbit [title] Listen to the sounds your rabbit makes. [step] Realize that your rabbit will probably be silent most of the time. As a prey species, rabbits have learned to be quiet in order to escape the notice of predators.\",\"endings\":\"['Some rabbits do make an occasional noise, either when very happy, scared, or as a warning. [substeps] This makes them very different from purring cats and dogs which use a whole vocal range to communicate.', 'This is a form of instinct by which the rabbit reacts to something approaching. As such, be sure to ask your rabbit why it is silent, and where it is hiding.', \\\"But that doesn't mean you can't be quiet in the same environment as one. [substeps] Pay attention to your rabbit's breathing and their normal breath.\\\", 'Therefore, it is important to keep track of your rabbit to determine the specific sounds it produces during a hunt. [substeps] Normally, rabbits will make response noises or grunts when chasing.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}