{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"090ef7c7-a233-5b08-a8b2-89926dff3be8","task_key":"default--validation--40676","task_revision_id":"1","upstream_id":"40676","short_description":"HellaSwag validation 40676","config":"default","split":"validation","body":"{\"activity_label\":\"Running a marathon\",\"ctx\":\"A large group of people are seen running down the street while many watch on the sides. several people\",\"endings\":\"['are then seen speaking to the camera while others run along and home in the background.', 'are seen speaking to the camera as more shots are shown of people running.', 'then run into one another and run in place.', 'in facial gear appear around the area where the men play while grabbing leads into people playing guitars and instruments.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}