{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ba053f88-af9f-5082-afc9-5f2568f53cdd","task_key":"default--validation--24048","task_revision_id":"1","upstream_id":"24048","short_description":"HellaSwag validation 24048","config":"default","split":"validation","body":"{\"activity_label\":\"Clean and jerk\",\"ctx\":\"A very slow playing video of a very strong man is shown as he picks up a barbell that has a lot of heavy weights on it, and he squats for a little bit with it at his chest. the man\",\"endings\":\"['bends slightly as he stands, leans back, lowers his gather, lifts it to his upper chest, while holding the barbell just above his chest with his lower body, as he sits on the barbell, continues to measure and then sets it back down.', 'does long bends to lift the barbells up over his head, then sends them off in the distance, but only the heavy weights land on the top of the wall on the right side of the room.', 'then picks up a third barbell and for a little bit he starts pushing it back and forth towards his knees so the weight there can be knocked off.', 'then stands up and holds the barbell to his chest.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}