{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"26f6e9db-d7d9-514e-9ab4-1a570db51967","task_key":"default--validation--26589","task_revision_id":"1","upstream_id":"26589","short_description":"HellaSwag validation 26589","config":"default","split":"validation","body":"{\"activity_label\":\"Playing badminton\",\"ctx\":\"Two people are seen playing badminton when one man speaks to the camera. The man moves his body around and demonstrates how to properly hit the birdie. he\",\"endings\":\"['continues to speak and demonstrate on how to play.', 'throws the ball more and faster while playing.', 'moves it all the way to the bag and continues to hit until the ball is clear and then lets it go.', 'continues to speak and advance to volley, volley and volley while the other two try to hit the ball back.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}