{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cf050df0-fbb0-5070-8662-ee2190a1d7ba","task_key":"default--validation--7924","task_revision_id":"1","upstream_id":"7924","short_description":"HellaSwag validation 7924","config":"default","split":"validation","body":"{\"activity_label\":\"Clean and jerk\",\"ctx\":\"The man then picks it up, squats, jumps to open his legs then quickly drops the bar and pushes the weight towards the middle of the bar and rests for a little bit. the man\",\"endings\":\"['then rounds his legs and gains the momentum to pick the weight back up and begins continuing to dumbell it again, this time squatting farther.', 'then lifts himself up to grab the bar using both legs and then he jumps onto his arms and holds it for a little bit missing his feet.', 'stands back up to test how he can do this and lets his legs crouch both ways.', 'goes back to the bar, struggles to pick it up, holds it above his head and then throws it to the ground where it bounces very hard.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}