{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e06f2473-9603-5a05-8649-265a937e6496","task_key":"default--validation--47467","task_revision_id":"1","upstream_id":"47467","short_description":"HellaSwag validation 47467","config":"default","split":"validation","body":"{\"activity_label\":\"Cutting the grass\",\"ctx\":\"A young boy is pushing a lawn mower forward and away from the camera on a very large yard of green grass. The boy turns the law mower and mows towards the left of the field, then down. the boy\",\"endings\":\"['turns to the right and continues raking as a man and three girls are combing the grass behind him.', 'stops mowing his lawn and turns around cutting trunk and then walks to the top of the lawn and back.', 'turns it once again and mows towards the right until he reaches the area where he first started.', 'picks up a large short stick and attaches it to the front of the lawn mower.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}