{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e6e08a8d-235f-5c73-b775-44925d30fc88","task_key":"default--validation--42440","task_revision_id":"1","upstream_id":"42440","short_description":"HellaSwag validation 42440","config":"default","split":"validation","body":"{\"activity_label\":\"Spinning\",\"ctx\":\"These people are pedaling using only one leg and they're standing in the middle of the bike with the other one holding their hands up. next they\",\"endings\":\"['do a different exercise where they put one foot on the pedal, while they have one foot out and they bounce up and down.', \\\"take each other's hands and give some sort of dance move on the brushes and they are using it to twirl their bodies and jump with the hands up.\\\", 'turn with their bricked hands and do a u-turn and then start their actions again and again.', 'switch to standing using two arms to balance themselves.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}