{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"3001c8ad-1e5b-519c-b61c-8c6d584a6afd","task_key":"default--validation--1821","task_revision_id":"2","upstream_id":"1821","short_description":"HellaSwag validation 1821","config":"default","split":"validation","body":"{\"activity_label\":\"Tennis serve with ball bouncing\",\"ctx\":\"He starts with his first serve where the ball hits the net. Then he takes another ball, bounces it and serves again. the ball\",\"endings\":\"['returns with a third serve.', 'bounces again and hits the net multiple times.', 'goes over the net and falls on the other side of the court.', 'comes back to him without touching the net.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}