{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"abae31f5-7d14-5218-9fdc-8a1aabed669f","task_key":"default--validation--44596","task_revision_id":"1","upstream_id":"44596","short_description":"HellaSwag validation 44596","config":"default","split":"validation","body":"{\"activity_label\":\"Making a lemonade\",\"ctx\":\"One of the parents is talking to the news reporter about the fund raiser. Three more parents are standing are chatting as they sip lemonade. a picture the girl, hannah\",\"endings\":\"['is shown for whom the fund raiser was held.', ', and her friend, melanie, are smiling as a young girl steps on the stair stepper and rolls.', ', is showing is snowy day in an indoor field.', 'is showing then sets the toy part of the hula hoop towards the camera and a person comes on the screen to you her, they are putting a red ring on their finger then they start to hula hoop.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}