{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"271b48ed-7d2c-5bd0-bbe2-86ec4e04472d","task_key":"default--validation--11905","task_revision_id":"1","upstream_id":"11905","short_description":"HellaSwag validation 11905","config":"default","split":"validation","body":"{\"activity_label\":\"Plastering\",\"ctx\":\"He goes from the bottom up and grabs more paint on his tool. He is making progress across the wall pretty quickly the way he is doing it. he\",\"endings\":\"['is getting what looks to be expert help from someone and he is smiling a lot.', 'uses his tools to rip off the excess paint, then runs his back up along the wall.', 'makes it half way across the wall but the bottom of the wall is still white.', 'then brings the canvas a closer look.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}