{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e6513d67-dbb8-574a-b2e1-a110d6d9d2de","task_key":"default--validation--10553","task_revision_id":"1","upstream_id":"10553","short_description":"HellaSwag validation 10553","config":"default","split":"validation","body":"{\"activity_label\":\"Clean and jerk\",\"ctx\":\"An intro of titles eventually leads to an athlete holding a large amount of weights over his head. The same man does another lit with his weights while another man watches in the background. he\",\"endings\":\"['throws the weights down and throws his arms up in the air to celebrate.', 'then lifts himself up over the bar several times.', 'jumps into the pit afterwards and the audience cheers and cries high.', 'gains force and is able to do the heavy lifting with help and experience.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}