{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a7f4f268-b770-53be-8d79-08405a1335c6","task_key":"default--validation--42087","task_revision_id":"1","upstream_id":"42087","short_description":"HellaSwag validation 42087","config":"default","split":"validation","body":"{\"activity_label\":\"Fun sliding down\",\"ctx\":\"A mom is sitting at the top of a slide with her little baby. She is contemplating going down for a while. She finally slides down it with baby. when they\",\"endings\":\"['reach the end, they are let inside and there it is.', 'finish she just sits there for a while and goes to sleep afterwards.', \\\"'re done, she sets them down and holds them to her chest.\\\", 'reach the bottom she lets him go and he starts to walk around.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}