{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b454db68-39fe-59ba-a4cf-3c8267b4b5c8","task_key":"default--validation--43656","task_revision_id":"1","upstream_id":"43656","short_description":"HellaSwag validation 43656","config":"default","split":"validation","body":"{\"activity_label\":\"Using parallel bars\",\"ctx\":\"A gymnast is seen climbing onto a set of uneven bars and performing a routine while many watch on the side. he\",\"endings\":\"['twists and turns himself around while still continuing his routine and ends with the dummy falling onto the ground.', 'performs various flips and tricks and ends by jumping down and putting his arms up.', 'also flips up on the bars to jump off and walk away.', 'jumps off the bars and proceeds to perform several loops and flips on the uneven bars and ends with him speaking with another gymnast.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}