{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"12a1e2a7-30c1-5941-bf4a-25d49d916019","task_key":"default--validation--1322","task_revision_id":"2","upstream_id":"1322","short_description":"HellaSwag validation 1322","config":"default","split":"validation","body":"{\"activity_label\":\"Clean and jerk\",\"ctx\":\"A man is in the gym in tight he bends over picks up a weight over his head and drops it back down. He walks back and loosens up before walking back up and doing it again adding more weight. he\",\"endings\":\"['finally lifts the weight over his head over his own head.', 'turns his back towards the camera and continues talking to the camera adding more weight along with more weight on the end.', 'does this multiple times adding more and more weight to the rack.', 'walks back up and drops the weights back down.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}