{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d390e3ca-6cc6-5739-a2f2-cd04e594ee4f","task_key":"default--validation--38450","task_revision_id":"1","upstream_id":"38450","short_description":"HellaSwag validation 38450","config":"default","split":"validation","body":"{\"activity_label\":\"Playing violin\",\"ctx\":\"A man is shown indoors playing the violin as people look on. He starts to play and people seem entranced by it. he\",\"endings\":\"['continues on as the camera pans around and shows the rest of the room he is in.', 'pauses, picks up and shows off his violin as the people eat lunch beside him.', 'continues to play to people while donning a mask, blowing smoke rings, and taking a bow.', 'continues trying to make a good music while moving his body quickly, narrating the movements.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}