{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"9fa03e0d-4e67-5fc5-801e-22eb4b7ae462","task_key":"default--validation--1277","task_revision_id":"2","upstream_id":"1277","short_description":"HellaSwag validation 1277","config":"default","split":"validation","body":"{\"activity_label\":\"Layup drill in basketball\",\"ctx\":\"A man is seen speaking to the camera while a shot of a person playing basketball shows behind. the man\",\"endings\":\"['continues going around the basket while the camera is sitting in the back.', 'continues to speak and leads into more clips of him playing tennis.', 'pulls the ball up on the court while still speaking to the camera.', 'continues speaking as more shots are shown of the boy playing basketball.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}