{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"38353cf7-49e4-55e2-9d5a-c0cb962ae1e6","task_key":"default--validation--49491","task_revision_id":"1","upstream_id":"49491","short_description":"HellaSwag validation 49491","config":"default","split":"validation","body":"{\"activity_label\":\"Slacklining\",\"ctx\":\"A large red slack line is connected to two trees out in a park. a strong young man\",\"endings\":\"['is lifting heavy weights over his head.', 'is shown on the other side of the tree pulling the line to ensure that is the tightest it can be around the trunk of the trees.', 'takes a stunt high, letting the rope go and goes flying.', 'combs the slack line one handed at a time along the trees in a park and cuts system of starts with three never ending lines.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}