{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"87dca7d3-e538-5bac-b26d-1abfd5383545","task_key":"default--validation--14307","task_revision_id":"1","upstream_id":"14307","short_description":"HellaSwag validation 14307","config":"default","split":"validation","body":"{\"activity_label\":\"Clean and jerk\",\"ctx\":\"The man then bends at the knees and puts his hands on the bar portion of the weight. the man\",\"endings\":\"['then lifts the weight up and down and pushes the weight side to side.', 'then pulls himself several times up successfully.', 'looks behind him where a person is standing and then almost yells \\\" urgh! \\\" people look in front of them, shocked.', 'then lifts the bell above his head and then lets the bell fall to the floor where it bounces a bit upon landing as the man turns and walks away from the weight.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}