{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"874280c3-1bf4-59c3-b6e9-e2e98fbe36b7","task_key":"default--validation--14253","task_revision_id":"1","upstream_id":"14253","short_description":"HellaSwag validation 14253","config":"default","split":"validation","body":"{\"activity_label\":\"Tennis serve with ball bouncing\",\"ctx\":\"Then the person in white begins taking about more about the tennis serve and how to play effectively. the player\",\"endings\":\"['in gray tries more of the serve and fails every time.', 'in white returns his serve and keeps it.', 'continues to demonstrate the serves.', 'goes back and forth between the two balls as the crowd claps and the man in black takes his bell and lets fly until he hits the ball.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}