{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2404d8fb-2cdc-5761-860a-b6128b01bc25","task_key":"default--validation--32982","task_revision_id":"1","upstream_id":"32982","short_description":"HellaSwag validation 32982","config":"default","split":"validation","body":"{\"activity_label\":\"Clean and jerk\",\"ctx\":\"A group of photographers take pictures as a weight lifter shows up on stage and prepares his hands. He lifts a barbell at the knees, then pauses at his chest. he\",\"endings\":\"['drops his head and does a back flip before being shown doing actual weight lifting.', 'releases his weight and it fails to hit the bar as it falls to the floor.', 'completes the performance, and then walks off stage as a crowd applauds.', 'lifts the barbell over his head before dropping it back to the ground.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}