{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"511c7859-6756-5f30-a426-ba1dbc7e27ad","task_key":"default--validation--9842","task_revision_id":"1","upstream_id":"9842","short_description":"HellaSwag validation 9842","config":"default","split":"validation","body":"{\"activity_label\":\"Layup drill in basketball\",\"ctx\":\"A black screen appears and the white words the mikan drill are displayed and the man continues to talk and then demonstrates exactly how to do various mikan drills. the man\",\"endings\":\"['then dumps out their clean clothing and they are then seen washing their clothes in a dryer and then placing the dirty clothes on stage.', 'then grabs a second ball and continues to demonstrate advanced mikan drills with both balls.', 'continues talking, exercises, and demonstrating how to properly properly do his all distance but 11 hour stay wakeboarding on a playground that is attached to a mobile tree.', 'continues to talk and then walks back and fourth and demonstrates more mikan drills that involve using an electric drill to drill through a coconut in a board with a handle.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}