{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"04d728f9-0557-514f-8ace-9007f914b8bd","task_key":"default--validation--28576","task_revision_id":"1","upstream_id":"28576","short_description":"HellaSwag validation 28576","config":"default","split":"validation","body":"{\"activity_label\":\"Vacuuming floor\",\"ctx\":\"The man then takes out fries and dumps them on the floor and begins eating the fries off the ground. while he\",\"endings\":\"[\\\"'s eating he still twirls his stick around while talking.\\\", \\\"'s doing this another man comes and grabs a chair and sits next to him who mouths off the food and continues eating.\\\", 'is doing it, several people walking by stop to take pictures in amusement that he is eating off the ground.', \\\"'s eating, another kind of skinny and hairy woman walks into frame and begins to hold up movies she has made.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}