{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3e59ac71-579e-5f64-85cf-45f504b2f2e9","task_key":"default--validation--30354","task_revision_id":"1","upstream_id":"30354","short_description":"HellaSwag validation 30354","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to deal with racist parents [title] Catch the behavior in the moment. [step] In any difficult confrontation, people may feel attacked if you bring up incidents from the past. If your parents say something that comes off as racist or insensitive, talk to them about as soon as you can.\",\"endings\":\"['Focus on why they resent you and tell them you want to speak to them about it. [substeps] For example, you could say, \\\" i know that we have different cultural backgrounds, but i wanted to talk to you about a specific inappropriate behavior.', \\\"It's best to address such things in the moment but this isn't always possible. If you're in public, for example, it might be hard to catch.\\\", '[substeps] For example, if you come home from school past curfew because your parents are no longer there, say you stayed late to make up. Hopefully they will see this statement as a compliment.', \\\"By avoiding direct confrontation, you will avoid answering your parents' questions for your actions. It helps to know that you do not have to respond to their questions in a way that they would want you to.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}