{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2e7d1727-3116-551f-9df4-d22ef3b39097","task_key":"default--validation--43583","task_revision_id":"1","upstream_id":"43583","short_description":"HellaSwag validation 43583","config":"default","split":"validation","body":"{\"activity_label\":\"Relationships\",\"ctx\":\"[header] How to act like you are very social [title] Try to talk more about stuff you saw online or something that your friend sent you this makes you seem more active and social. [step] Be more knowledgeable about current topics and issues in the world that you can relate to with others. Become more aware of the world you live in.\",\"endings\":\"['Not only does this help you have more people around you, but is it also something that others will admire about you and can tell you about if you go around alone. It can also be something about the way other people treat you that you could have researched.', \\\"[title] Start looking around at the world from a specific angle, when you are hanging out in different places such as a cafe or eating lunch at a playground. [step] Don't reveal everything when you speak, just talk.\\\", \\\"[title] Be light and cheery, this makes people think that you are fun and outgoing. [step] Don't be gloomy and act bored.\\\", 'Do some research and see what has been published about social groups. Ask friends if they have ever gone out with.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}