{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"3a94253b-8c26-5bc8-88fc-2c35235022ab","task_key":"default--validation--386","task_revision_id":"2","upstream_id":"386","short_description":"HellaSwag validation 386","config":"default","split":"validation","body":"{\"activity_label\":\"Washing face\",\"ctx\":\"A black female is shown in a room with a black scarf around her head. Black spots on her faced are then zoomed in on and she begins to apply something on it. next\",\"endings\":\"[', she begins to brush her lips and make painted lips look just like a lipstick to the camera and ends with her closing gift and instruction showing her off at the end.', 'she is shown in various different locations and gets extremely close to a wall.', ', she is shown making multiple moans and sprays some kind of liquid down her face before hanging up.', ', she takes some spray and it makes her skin extremely oily and she continues to talk until the video is over.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}