{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bf8b3f3e-0f73-5d2d-ae4d-715149059715","task_key":"default--validation--27885","task_revision_id":"1","upstream_id":"27885","short_description":"HellaSwag validation 27885","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to deal with relatives you hate [title] Think about how you want to behave. [step] Before you spend time with this relative, take a few moments and decide how you want to behave. Perhaps you and this relative have gotten into arguments in the past.\",\"endings\":\"[\\\"Or maybe you don't like being around a relative you dislike. If you want to deal with these kinds of situations, the best thing to do is prepare beforehand, not just spend time in these situations.\\\", 'Ask yourself what started these arguments and if there are ways to avoid getting into an argument this time. [substeps] You may be proud to be atheist, but your aunt may truly believe that being atheist will land you in hell.', \\\"Alternatively, you and this relative may have used certain words to make fun of you. [title] Don't feel guilty for feeling afraid.\\\", \\\"Perhaps, you wanted to start a family but your cousin couldn't do it right now. The decision is up to you, so choose the mindset that is most realistic for the situation and that makes you happy.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}