{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5c13600d-991f-5434-b880-e41bf58fe699","task_key":"default--validation--48008","task_revision_id":"1","upstream_id":"48008","short_description":"HellaSwag validation 48008","config":"default","split":"validation","body":"{\"activity_label\":\"Youth\",\"ctx\":\"[header] How to annoy your older brother without getting in trouble [title] Talk loudly. [step] Noise is arguably the most important contributing factor towards annoyance. Anything you say can be made annoying based on volume alone.\",\"endings\":\"[\\\"[substeps] People who are vocal have a better chance of getting rid of annoying noise than those who are mostly silent. Laughing to yourself when you're having a really bad day just doesn't work.\\\", \\\"Whenever you're attempting to be annoying, make an effort to keep your speaking voice loud. Try to stay clear of outright screaming, however, as it's not sustainable.\\\", 'The louder your voice, the more annoying your younger brother will be. Loud voice sounds can remain the same throughout life, as long as you give them the proper volume in order to make the conversation last as long as possible.', \\\"People that are talkative may not realize in some circumstances that they can drop volume, and being loud makes them less inclined to listen. Laughter, while appropriate, is perhaps the greatest and most effective way to annoy someone who doesn't like you.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}