{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7edf30f8-074a-53c6-9c0c-f7fd44b74893","task_key":"default--validation--25074","task_revision_id":"1","upstream_id":"25074","short_description":"HellaSwag validation 25074","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to deal with difficult in laws [title] Respect your in-law's attachment to your spouse. [step] Your spouse is their little boy or girl, and always will be, no matter how grown up. Especially if you have your own young children, think about how difficult it will be to \\\" let them go \\\" as adults someday.\",\"endings\":\"[\\\"You have to be there to support them as they deal with complicated situations. It is also a good idea to allow them to have time to themselves occasionally as they adjust to their parent's new behavior.\\\", \\\"Remember that a toddler can't have everything. You have to make an effort to respect their dependability at this young age.\\\", \\\"[substeps] Often what appears as undue criticism from an in-law comes from a parent's protective instinct. Many parents feel that their child's partner has to prove that he/she is worthy.\\\", '[substeps] If you happen to get divorced, your spouse will be most likely to want to remain behind, especially if their children have grown up already. [title] Realize that most people have different bonds with their in-laws.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}