{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e37d2a6b-ddf0-5fe0-854b-be30cb6d6151","task_key":"default--validation--16770","task_revision_id":"1","upstream_id":"16770","short_description":"HellaSwag validation 16770","config":"default","split":"validation","body":"{\"activity_label\":\"Rock climbing\",\"ctx\":\"There's another young man who is demonstrating how to climb up the wall while holding onto the wall mounts. the man in the green shirt\",\"endings\":\"['explain the technique as he points to the wall to show the significance of each rock and its height.', 'is in the same room climbing the wall mounted goal and he also finds out how to do the trick while holding the wall mounted goal.', 'gets on one of the mounts and hoists himself up and starts doing tricks on the impressive piece of machinery.', 'is climbing as the wall behind him is covered in snow.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}