{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"43778871-252d-5782-a619-a5339880d12e","task_key":"default--validation--44109","task_revision_id":"1","upstream_id":"44109","short_description":"HellaSwag validation 44109","config":"default","split":"validation","body":"{\"activity_label\":\"Shuffleboard\",\"ctx\":\"A young boy is seen riding along a skateboard followed by several more shots of people riding on boards. the boys\",\"endings\":\"['speak to one another while driving and ends with the boy riding away and many shots of the bike riders riding on the boards.', 'continue riding around on the board while a helicopter appears above them with one around his neck and another around his waist.', 'perform several tricks while the camera pans into the sky and move up and down the track.', 'jump all over the board and other people stand on the sides smiling and laughing.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}