{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"239b6345-3ae3-55d0-b1fc-f047f306c51c","task_key":"default--validation--32672","task_revision_id":"1","upstream_id":"32672","short_description":"HellaSwag validation 32672","config":"default","split":"validation","body":"{\"activity_label\":\"Ping-pong\",\"ctx\":\"A man seated in front of a tennis net turns around and faces his opponent, with a coach and camera man looking on. as they\",\"endings\":\"['are watching, the man doubles over in pain.', 'remain facing each other, the other begins to dribble while the man in front gets fet by the ball and the man behind him helps the man across the court.', 'both rise we see that they are actually heading to a table tennis game, the table on an actual tennis court.', \\\"continue to play, the goalie chops the goalie's head, then the man ends their tennis match and falls to the ground on top of his friend who cheers loudly.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}