{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ff97dcdd-801a-5e64-b4dc-dd9c62fd428b","task_key":"default--validation--43112","task_revision_id":"1","upstream_id":"43112","short_description":"HellaSwag validation 43112","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to be a successful muslim husband [title] Talk with your spouse. [step] Ask about their feelings, and share yours with as well. Be honest and open about your joys and fears.\",\"endings\":\"['Encourage your spouse to express and discuss these as well. [substeps] If one or the other feels uncomfortable discussing their feelings, keep the focus on both of you.', '[title] Serve the blessings god provides to us. [title] Inspire your spouse to follow a range of virtues.', 'If your husband seems offended, frustrated, or gives signs that he is upset, it may be a sign of a deeper problem. Be sure to ask him things like, \\\" is it okay to say things in arabic? \\\" and \\\" have you ever noticed when i...\\\".', 'Listen to what they have to say, and ask follow-up questions when they are telling you something. [substeps] Make time for conversations even on busy days.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}