{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"6a7df164-939f-5297-89ee-64167f6e9eae","task_key":"default--validation--2583","task_revision_id":"2","upstream_id":"2583","short_description":"HellaSwag validation 2583","config":"default","split":"validation","body":"{\"activity_label\":\"Hand washing clothes\",\"ctx\":\"Initially he starts to wash the clothes with both hands. Afterwards he adds more soap using two different bottles to the sink. subsequently he\",\"endings\":\"['keeps washing the clothes as he talks to the camera.', 'adds more soap to the washing machine and then proceeds to swirl and rinse the clothes.', 'ends by miming drinking the water in the sink.', 'grabs fabric to dry the clothes and shows how they fit him comfortably.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}