{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"40ad3e9b-2d5a-5c76-bcdb-b050b8b464b7","task_key":"default--validation--31917","task_revision_id":"1","upstream_id":"31917","short_description":"HellaSwag validation 31917","config":"default","split":"validation","body":"{\"activity_label\":\"Clean and jerk\",\"ctx\":\"A man walks around while adjusting a belt, with many other people in the room. The man walks through a tunnel flanked by several other men. the man\",\"endings\":\"['meets a young man, who is pole volting to the ground and hopping back to the platform after he finally pole volts himself to the ground.', 'walks in abbens circle on the floor.', 'then appears outdoors swinging a large watermelon on his hands.', 'gets on a stage and powders his hands.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}