{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e5fb02ec-e92d-54b9-a2ff-083c0663f257","task_key":"default--validation--1129","task_revision_id":"1","upstream_id":"1129","short_description":"HellaSwag validation 1129","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to keep relationships healthy within your family [title] Exchange daily stories. [step] As soon as you get home, show that you're excited to reunite with them by immediately speaking with your family members about each other's day. Ask your family members how their days were, and share a story from your own day with them.\",\"endings\":\"[\\\"Sharing the narratives of our daily lives allows us to be more in tune with our loved ones' lives. [substeps] Seek out younger family members as soon as you get home.\\\", 'This will allow you and your family to bond, and even set the mood for the reunion. [substeps] Making conversation with family can also help bring some good back-to-school memories to you.', \\\"You will develop a strong relationship despite being separated for awhile. [substeps] If your significant other's day is coming up, tell them how much you enjoy spending time together.\\\", \\\"Avoid doing so until the adults present tell you to leave. [substeps] If you've never seen your parents, write a short, sketch book about where and how each of you went to school or your future plans.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}