{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"77e29b86-a068-5bfd-94be-267f3abd77f3","task_key":"default--validation--16769","task_revision_id":"1","upstream_id":"16769","short_description":"HellaSwag validation 16769","config":"default","split":"validation","body":"{\"activity_label\":\"Rock climbing\",\"ctx\":\"A young man wearing a green shirt is talking about the techniques involved in rock climbing. Two young men are seen climbing the wall while being suspended on ropes. there\",\"endings\":\"['are several other people showing the techniques in the audience.', 'is a third man wearing a blue shirt climbing the wall from a boards down and skipping down the wall up a rock and landing on his feet at one point.', 'is also a young boy shimmy on the rope and adjust his feet while holding on.', \\\"'s another young man who is demonstrating how to climb up the wall while holding onto the wall mounts.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}