{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"5dbe4b6f-ef97-5307-8e4d-7db4fe3469fc","task_key":"default--validation--1189","task_revision_id":"2","upstream_id":"1189","short_description":"HellaSwag validation 1189","config":"default","split":"validation","body":"{\"activity_label\":\"Rollerblading\",\"ctx\":\"A pilates instructor on skates explains how to strap your risk protectors tight and shows that you need knee pads to protect your knees. the lady\",\"endings\":\"['starts with two small straps and it puts her into a squatting position with feet fully placed over her knee and her hands suspended and ready.', 'then explains the skates movement and how to use the breaks that is in the back of the skates.', 'then puts lotion in both of her hands and rubs them along your knees while the instructor washes them.', 'shows how to harness and strap your weight on properly and she moves away from the mounting as she reaches for her left knee.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}