{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0549d9f3-b411-543f-a647-87f9bdd8b5cb","task_key":"default--validation--35990","task_revision_id":"1","upstream_id":"35990","short_description":"HellaSwag validation 35990","config":"default","split":"validation","body":"{\"activity_label\":\"Using the rowing machine\",\"ctx\":\"A man in a blue tank top and a man wearing a black tank top work out on the rowing machines with the man wearing the orange shorts. The men work out running on a treadmill. the man wearing a grey t - shirt\",\"endings\":\"['focuses on exercising so he can focus better.', 'and the man wearing the black tank top jump on and off of the pile of gym mats.', 'turns a dial on the machine and the man underneath puts one foot on the 18 an the man jumps on the machine to work out.', 'walks backstage talking to the camera and he joins the man wearing the black tank top.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}