{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a2805823-8326-5523-a561-471fb785a5ee","task_key":"default--validation--3683","task_revision_id":"1","upstream_id":"3683","short_description":"HellaSwag validation 3683","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to be a romantic husband [title] Tell your spouse how happy you are that you're together. [step] There are about a bajillion ways to do so, and the dialogue you share with your spouse will likely dictate the specifics. That said, there are few things you can say that have particularly romantic potency.\",\"endings\":\"['[substeps] Tell your spouse you love them every day. Add to your \\\" i love you, \\\" with something like, \\\" i\\\\'m so glad we married one another.', '[substeps] For example, you could say something like, \\\" i am so glad to have you in my life. Having you around will make my day even better, \\\" \\\" my relationship is moving on and i am already seeing someone.', \\\"The most direct way to say what you're feeling is to be specific. Being specific is the smoothest way to get across your love.\\\", 'Here are some people to thank for : [substeps] \\\" my husband has so much fun with you on our anniversary. You\\\\'re so incredibly wonderful in his arms.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}