{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"72c6134a-b064-5e01-b28e-b99214c5b7e4","task_key":"default--validation--26889","task_revision_id":"1","upstream_id":"26889","short_description":"HellaSwag validation 26889","config":"default","split":"validation","body":"{\"activity_label\":\"Clean and jerk\",\"ctx\":\"A man in a weightlifting outfit stands in front of a small group of people and lifts a heavily weighted bar bel from an elevated platform, above his head. A man in a body suit bends before a bar bell with his hands on the bar. the man\",\"endings\":\"['then lifts the barbell above his head before changing leg stance to better support the weight and finally allowing the bell to drop to the floor from the height of his head and bounce as helpers rush the stage to get the bell.', 'lifts, puts, and cheers.', 'performs simple form lifting and lets go immediately.', 'pumps his gloved hand then spins himself around to bend the bar twice.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}