{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1ee0cb67-961a-5b9d-a40f-4fc59b8d82ef","task_key":"default--validation--22853","task_revision_id":"1","upstream_id":"22853","short_description":"HellaSwag validation 22853","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to get other guys to stop staring at your pretty wife [title] Get your wife's opinion. [step] Value how your wife feels about the situation and get an understanding of how if affects her. If she is bothered by the stares, discuss whether or not you should intervene on her behalf.\",\"endings\":\"[\\\"If you feel the stares are a direct threat, try to think about your wife's feelings. Most of all, give her the benefit of the doubt-no guy wants to get a first-degree stare.\\\", 'If she says \\\" i don\\\\'t feel like i need your attention right now, \\\" then you can bring it up with her and let her ask her opinions. [substeps] Try to use humor to break the ice.', '[substeps] If you start to think her boyfriend is doing this, report it back to him. If he is only doing this to steal her attention, admit that you feel sorry for him.', '[substeps] If you have addressed this issue in the past, then you may already know how your wife feels about the attention that she receives. If this is the case, make a decision in advance on how you will respond so that you do not overreact.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}