{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"076d2d6c-b676-5c22-b0f9-3d77d8d74469","task_key":"default--validation--41237","task_revision_id":"1","upstream_id":"41237","short_description":"HellaSwag validation 41237","config":"default","split":"validation","body":"{\"activity_label\":\"Washing face\",\"ctx\":\"A dog is close to the camera, play fighting with another dog. They jump off a man's lap, then back on again as they continue to play. the man\",\"endings\":\"['starts throwing the frisbee to his dog and he jumps through the air.', 'is on his feet, smiling as he receives a trophy.', 'puts his glasses back on as the dogs calm down and lie down on his lap.', 'gets up, jumps on the back of the dog, and they continue to play again.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}