{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"25605094-3822-5049-904d-063c3b905794","task_key":"default--validation--34803","task_revision_id":"1","upstream_id":"34803","short_description":"HellaSwag validation 34803","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to get your spouse to stop a bad habit [title] Choose the right moment to draw attention to your spouse's habit. [step] Don't try to describe the problem to your spouse when both of you are rushed and heading out the door to work in the morning. Similarly, don't try to have a serious conversation with your partner when you are trying to cook dinner, dress your child, or watch tv.\",\"endings\":\"['Asking him to stop doing something that is both dangerous and detrimental may discourage your spouse from doing the thing that you want him to do. Instead, try to present the issue in a positive light.', 'These conversations may involve chores, meetings, etc. They could appear out of the blue, or may include questions or reminders from your partner about your lifestyle choices.', \\\"[substeps] Avoid having a conversation about your partner's habit in public or at their place of work. [title] Bring up the subject of the habit in a gentle way.\\\", 'Avoid this conversation during your morning commute with your spouse and if it is in your home. [title] Find a way to mark the end of your morning commute.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}