{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"93360c0f-056e-5345-bce2-e595c6ed846e","task_key":"default--validation--44115","task_revision_id":"1","upstream_id":"44115","short_description":"HellaSwag validation 44115","config":"default","split":"validation","body":"{\"activity_label\":\"Hula hoop\",\"ctx\":\"She goes to the grass and she begins hooping in circles, she is having a really good time dancing and hooping. then\",\"endings\":\"['she starts to quick dance on the grass of the room.', ', other people are dancing around and hooping also, doing tricks and just enjoying the day.', 'her mom comes back and she begins hooping herself doing cardio and her mom comes to help.', 'she takes a break and they both finish dancing and shake hands.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}