{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a7a6c186-cbff-5886-8006-76f63e46588f","task_key":"default--validation--29309","task_revision_id":"1","upstream_id":"29309","short_description":"HellaSwag validation 29309","config":"default","split":"validation","body":"{\"activity_label\":\"Baton twirling\",\"ctx\":\"Then, the young man spins the stick on his neck, after makes moves with the hands. After, the young man jumps over the stick, then spins his body stand in one leg. next\",\"endings\":\"[', the young man hops over the floor and make different steps.', ', the young man kick the wooden table and standing on the feet.', ', the young man jumps over the stick to reach the catch on the floor.', ', the young man performs a mix of moves.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}