{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"70e6b6d7-c03c-51b1-9810-a3aea63e7dee","task_key":"default--validation--9640","task_revision_id":"1","upstream_id":"9640","short_description":"HellaSwag validation 9640","config":"default","split":"validation","body":"{\"activity_label\":\"Skiing\",\"ctx\":\"This man is skiing in the snow while holding a selfie stick recording himself. First he starts at the start line where racers are who participate in skiing competitions. then he\",\"endings\":\"['goes to another line where skiers are continuing to sled in the snow.', 'records himself on a snowy slope, then he starts skiing down lemkes.', 'continues skiing until he reaches the end and only people who are very loud speak to him.', 'shows himself in the snow skiing down the snowy slopes.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}