{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e8cbb5d5-4428-5741-a399-ab32d03e09dc","task_key":"default--validation--1155","task_revision_id":"1","upstream_id":"1155","short_description":"HellaSwag validation 1155","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to deal with your brother in law's jealous wife [title] Stand up for yourself. [step] You are a part of the family now. Your husband chose you to be his wife.\",\"endings\":\"['He has taken too long to explain himself to you about the situation, but now he wants to know that you care for yourself. You should be strong and proud of yourself.', 'Unfortunately, as a family, we all make mistakes. Your actions, though, can hurt your wife.', \\\"Now doesn't mean you have to have been blamed for everything. Instead, show him the respect he deserves by taking charge and acting confidently about everything that happens in your life.\\\", 'You have every right to be included as a part of this new family. Building a relationship with your in-laws is very important so do make the effort to visit often and get to know them and allow them the opportunity to get to know you.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}