{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"118f13ef-df02-5d7a-a28b-427a41bf2943","task_key":"default--validation--14306","task_revision_id":"1","upstream_id":"14306","short_description":"HellaSwag validation 14306","config":"default","split":"validation","body":"{\"activity_label\":\"Clean and jerk\",\"ctx\":\"A man stands indoors, on a platform, in front of three onlookers, and lifts a heavily weighted bar bell. A man approaches a barbell on the floor and stands before it preparing to lift it. the man\",\"endings\":\"['lifts the weight bar suspended in the air on the platform and turns towards the onlookers.', 'lifts the bar and is seen demonstrating how he performs, pausing on each bar to measure the weight.', 'then bends at the knees and puts his hands on the bar portion of the weight.', 'then lifts the bell on his shoulders, leans lean back, puts his arms behind his head and lifts it slightly.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}