{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"10e11893-dfd5-57cb-a0d3-9cfcfa1c90a4","task_key":"default--validation--5014","task_revision_id":"1","upstream_id":"5014","short_description":"HellaSwag validation 5014","config":"default","split":"validation","body":"{\"activity_label\":\"Getting a haircut\",\"ctx\":\"A woman is lying on a bed with her hair spread. There are several images of her from behind with very long hair. we\",\"endings\":\"['see several women getting their hair cut with scissors.', 'see a man shaving her cheek.', 'then see numerous images of her with her hair cut off to shoulder length.', 'then see her from the backs as she is getting ready with her hair down.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}