{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e6891fbb-6496-5ff2-96da-6563da49dcf2","task_key":"default--validation--48177","task_revision_id":"1","upstream_id":"48177","short_description":"HellaSwag validation 48177","config":"default","split":"validation","body":"{\"activity_label\":\"Clean and jerk\",\"ctx\":\"A very thick man walks slowly into a a large room filled with people and he stands in front of a barbell with red and blue weights on both ends. the man\",\"endings\":\"['sits the trainer down on the ground next to the barbell and he begins to make circles on the weighted bar.', 'takes a few breaths then reaches down and puts his hands on the barbell.', 'then steps on the weights and walks slowly into the middle and begins to lift the weights up and down over his head.', 'begin to large lift on the bar.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}