{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d96d69f1-9187-5d7e-ae2c-861d856f38b4","task_key":"default--validation--7517","task_revision_id":"1","upstream_id":"7517","short_description":"HellaSwag validation 7517","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to be a favourite niece or nephew [title] Be respectful. [step] This is a crucial part of being a good niece or nephew. Be respectful of your aunt/uncle's beliefs and personality.\",\"endings\":\"[\\\"Be prepared for this as they've a different view than you. When your aunt/uncle (or anyone close to them) raises the subject, you can speak nicely and listen to their points without any resistance.\\\", 'It is also important never to argue rudely with them. Show acceptance of their opinions by not rebelling.', \\\"Don't lead every conversation down the wrong path or your aunt/uncle may get annoyed! [title] Respect their needs. [step] People are often deprived of the type of life they want or the qualities that show through.\\\", \\\"You cannot speak as though you know them well, as if you don't understand their views and goals. Be kind and thoughtful, and never cut yourself or your family off in order to please your aunt/uncle.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}