{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7c230929-9b1c-5e7f-adba-4511939959f1","task_key":"default--validation--40130","task_revision_id":"1","upstream_id":"40130","short_description":"HellaSwag validation 40130","config":"default","split":"validation","body":"{\"activity_label\":\"Spinning\",\"ctx\":\"A close up of an exercise bike is shown with a person walking into frame. The person climbs on the bike and begins riding furiously. the person\",\"endings\":\"['continues riding on the bike and jumps off in the end to gesture to the camera.', 'rides as he is moving backwards all the way back while rubbing the bike against the bike.', 'continues to ride the bike and the camera man pans to the wrong side.', 'stops and puts his bike down.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}