{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7b7ef74c-5056-516c-8f20-d2c10ae2f572","task_key":"default--validation--9322","task_revision_id":"1","upstream_id":"9322","short_description":"HellaSwag validation 9322","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to swallow a pill [title] Eat bread. [step] If you are trying to take a pill and just can't seem to get it to go down, try using a piece of bread. Pull off a small piece of bread and chew it until you are ready to swallow.\",\"endings\":\"['This works especially well if you have normal blood sugar and you are trying to get it down. [title] Take an over the counter or capsule.', 'Before you swallow, take your pill and stick it in the mass of bread in your mouth. Once you close you mouth, swallow the food with the pill inside.', 'Take your time chewing, not swallowing the raw bread. [substeps] Always grab a piece of bread before you start swallowing the pill to prevent yourself from biting your fingers or getting bites.', '[title] Sprinkle salt in to avoid long-term health issues. [step] This can be tricky at times, so be okay.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}