{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d6f58bb4-61b2-5905-8c38-bc13d55bf107","task_key":"default--validation--28728","task_revision_id":"1","upstream_id":"28728","short_description":"HellaSwag validation 28728","config":"default","split":"validation","body":"{\"activity_label\":\"Clean and jerk\",\"ctx\":\"A woman is show in several different location lifting weights above her head and jumping her legs out. the woman\",\"endings\":\"['continues lifting up the weights over her head while various text instruments pan across the screen.', 'ends with her also jumping her legs down.', 'continues going up as various discs are thrown up and throughout her body while the camera follows.', \\\"continues to run up and down the trainer's world and ends by dropping keys and jumping back down.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}