{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7a28eb8a-5d55-593f-8fdc-1123cd4c146d","task_key":"default--validation--41917","task_revision_id":"1","upstream_id":"41917","short_description":"HellaSwag validation 41917","config":"default","split":"validation","body":"{\"activity_label\":\"Cutting the grass\",\"ctx\":\"A shirtless man with a machete walks out into the front yard. He starts hitting the grass repeatedly over and over cutting the grass. instead\",\"endings\":\"['of spinning around on the grass, he starts again and ends by walking away.', 'of using a lawnmower he is cutting the grass with a machete.', 'of stopping an gored up woman walks out from behind the house looking at the man.', 'of cutting corners, the man starts to spread an shaped sheet out covering the entire yard.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}