{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e4671251-a644-52e3-a27a-fb7a10d79a68","task_key":"default--validation--3744","task_revision_id":"1","upstream_id":"3744","short_description":"HellaSwag validation 3744","config":"default","split":"validation","body":"{\"activity_label\":\"Cutting the grass\",\"ctx\":\"They grab some pieces and use the scissors to cut it up. Then throw the grass they cut back behind them. they\",\"endings\":\"['then start moving their pieces as they are pushing them through the grass.', 'pick up another tree back behind them.', 'do several more stunts as they fight for the kite.', 'are going at it kind of quickly, they seem like very bored little kids with nothing else to do.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}