{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"204c1ca2-4a3f-59e3-a068-45a1f63b00eb","task_key":"default--validation--5814","task_revision_id":"1","upstream_id":"5814","short_description":"HellaSwag validation 5814","config":"default","split":"validation","body":"{\"activity_label\":\"Tumbling\",\"ctx\":\"A man is inside of a gym. He starts doing flips inside the gym. He does many tricks as he jumps in the air. he\",\"endings\":\"['lands on his feet after the tricks.', 'stops performing many tricks back and fourth.', 'continues to do the splits as he sports his body from various angles.', 'does more flips from the outside and on inside the gym.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}