{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"382c5b14-1e4a-575e-8225-6501c7c91b54","task_key":"default--validation--40046","task_revision_id":"1","upstream_id":"40046","short_description":"HellaSwag validation 40046","config":"default","split":"validation","body":"{\"activity_label\":\"Clean and jerk\",\"ctx\":\"A weight lifter is bend to raise a weight sitting on the floor wile a woman watch him. then\",\"endings\":\"[', the man lifting weight jumps in air while the woman watch him, then he drops the weight on the floor.', 'he lifts the weight till a third man prepares for weightlifting.', ', the man raises the weight to rest on the shoulders, and then above the head.', ', the man lift the weight, and then the woman talks.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}