{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e97dc424-5159-5eea-a6e9-b8099d23c9fb","task_key":"default--validation--36863","task_revision_id":"1","upstream_id":"36863","short_description":"HellaSwag validation 36863","config":"default","split":"validation","body":"{\"activity_label\":\"Cutting the grass\",\"ctx\":\"A man is outside dressed in a blue hoodie and burgundy pants pushing a lawn mower across a small patch of grass. as he\",\"endings\":\"['is working, he comes across two trampled grass clippings under a plastic fence.', 'attempts to push the dirt machine through any remaining patches of over grown grass, he ends up mowing the lawn all on his own.', 'mows the lawn, the camera is on him helping as he mows while the camera pans left and right.', 'turns, he has difficulty because the back yard has a trampoline and several blocks around it.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}