{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"c1bdd5d4-e60d-59ac-a12d-6e6c2600d531","task_key":"default--validation--2582","task_revision_id":"2","upstream_id":"2582","short_description":"HellaSwag validation 2582","config":"default","split":"validation","body":"{\"activity_label\":\"Hand washing clothes\",\"ctx\":\"A young man is washing clothes inside a bathroom sink as he talks to the camera. Initially he starts to wash the clothes with both hands. afterwards he\",\"endings\":\"['adds more soap using two different bottles to the sink.', 'washes the other clothes in the sink and several more people walk by.', 'shows his face by grimacing due to the stiff air stream and looks in the mirror to change something for his face.', 'goes back to using his hands to wash the clothes gradually.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}