{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"219563db-4ca6-5815-96ac-d96309c45d56","task_key":"default--validation--32177","task_revision_id":"1","upstream_id":"32177","short_description":"HellaSwag validation 32177","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to handle a dramatic sister in law [title] Realize that you may be dealing with a drama queen. [step] The dynamics of a sister-in-law (s-i-l) are complex at the best of times but no more so if she has spent much of her life causing her immediate family to run to her beck and call. The drama queen thrives off drama and having everyone pay attention to her as a result.\",\"endings\":\"['She is bold enough to make it seem like the answer to all her problems is somehow \\\" none of your business. \\\" [title] Know that you are not alone.', 'She uses her powers to get her way and leads the world around her. [title] Realize that even though you may not be very close, dynamics work.', '[title] Think of how important she really is. [step] A major component of a drama queen personality is always thinking of how much women, children and other people value her.', '[substeps] Sit back at your next family occasion and simply watch. Notice how she interacts with her family members, and how they in turn react back.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}