{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"17d0099c-3238-5350-8bc8-c4b1ae97a6f3","task_key":"default--validation--1012","task_revision_id":"2","upstream_id":"1012","short_description":"HellaSwag validation 1012","config":"default","split":"validation","body":"{\"activity_label\":\"Hitting a pinata\",\"ctx\":\"A man is seen walking closer to a pinata while a large group of people stand around him. the man\",\"endings\":\"['shows off various tapers while swinging at the pinata and throwing them out to the crowd.', 'then throws the pin while others watch on the sides.', 'spins around several times and throws the stick.', 'misses the pinata and hits a person behind him, shown again in slow motion.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}