{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8104f15c-5329-5496-9408-b3dece67f7c8","task_key":"default--validation--16955","task_revision_id":"1","upstream_id":"16955","short_description":"HellaSwag validation 16955","config":"default","split":"validation","body":"{\"activity_label\":\"Clean and jerk\",\"ctx\":\"The woman stands still for a short while in her blue/white outfit. Then she lifts the heavy yellow weight and someone who's watching her shows peremptory by yelling. she\",\"endings\":\"['picks it back up and lifts it again, guzzling the last of the shiraz.', 'drops the weight and the face of female trainer starts into the screen.', 'holds it up for a good second and then drops it back down to the ground.', 'slowly lifts the weight up she long the overweight weight over her head, then drops the weight to the ground.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}