{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e78d8dc3-3404-5dcd-8689-c7b312325cca","task_key":"default--validation--16530","task_revision_id":"1","upstream_id":"16530","short_description":"HellaSwag validation 16530","config":"default","split":"validation","body":"{\"activity_label\":\"Clean and jerk\",\"ctx\":\"Two men are standing in the corner around a stand putting powder on their hand. One man then walks onto the stage and positions himself to lift up a weight. after a long time\",\"endings\":\"[', the man removes the weight.', ', the man picks up the weight in a squat, bends his knees several times and finally jerks it over his head.', ', the one with the mask walks out onto the stage and sits next to the man with the mask and begins talking.', 'he leaves the stage to begin the match.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}