{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b2ea7f99-2bcb-5fe6-8134-765270388e62","task_key":"default--validation--13277","task_revision_id":"1","upstream_id":"13277","short_description":"HellaSwag validation 13277","config":"default","split":"validation","body":"{\"activity_label\":\"Using parallel bars\",\"ctx\":\"A man steps up to a set of uneven bars while a man does flips behind him and begins performing a gymnastics routine. the man\",\"endings\":\"['flips and spins himself around on the bars and ends by jumping down and waving his hands up.', 'releases the bars and then the gymnast catches both of them on his hands.', 'does his routine while shaking the bar and jumping, it goes into a routine until break time.', 'jumps over and around four bars in the air while jumping far back over the remaining bars and landing on his butt.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}