{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2c468a6b-7a26-5712-bf8e-831794aefbbc","task_key":"default--validation--1279","task_revision_id":"1","upstream_id":"1279","short_description":"HellaSwag validation 1279","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to reduce sibling rivalry (for parents ) [title] Ignore minor offenses like teasing. [step] Sometimes, they will bicker simply to get your attention. Refrain from giving them attention for negative behavior.\",\"endings\":\"['For example, if they say something like \\\" i\\\\'m so stupid for leaving all of my chores at home \\\" don\\\\'t give them attention until they stop joking around. [substeps] Does this sort of behavior happen often? This may prevent the rivalry from getting worse.', 'Ignoring minor offenses like teasing will only strengthen the relationship and make things worse. [substeps] Explain this, \\\"\\\\' oops\\\\', i won\\\\'t be the source of the teasing.', 'Try your best to stay out of it. Removing yourself from the room when they start to squabble will send them the message that their arguing does not garner any attention from you.', 'Other times, they might taunt you intentionally in order to get you to do something more bad. [substeps] If they are teasing you, politely ignore them by saying that you need to talk to a parent or go for a walk.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}