{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bd55088d-1b06-5da2-869b-e6d0907356cf","task_key":"default--validation--40668","task_revision_id":"1","upstream_id":"40668","short_description":"HellaSwag validation 40668","config":"default","split":"validation","body":"{\"activity_label\":\"Playing violin\",\"ctx\":\"She continues talking to the camera followed by her playing the violin. She takes her hair down and continues talking to the camera and showing various violin moves while singing. she\",\"endings\":\"['makes secret routines before walking away.', 'stopping to play a couple of notes and ending by bowing to the camera.', 'is crying again, and we can see her holding her hands up.', 'once again puts her hair up and continues with her playing.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}