{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2d7004f0-72b9-501d-b87a-0d3e810d1694","task_key":"default--validation--35766","task_revision_id":"1","upstream_id":"35766","short_description":"HellaSwag validation 35766","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to accept a marriage proposal [title] Don't ruin the surprise. [step] If you believe that your partner plans to propose to you, give him or her the chance to carry out a plan. Don't try to rush the process along.\",\"endings\":\"['Let him or her take the lead, but try to appear as engaged as possible as you challenge your partner to compromise or change his or her mind. Avoid waiting until the timing is perfect to truly ask.', 'Be thoughtful and allow your partner to decide the next level of urgency (the best time for you and for your relationship). [title] Have your heart set on the proposal.', \\\"Don't give any indication that you expect it. [substeps] If you find out for certain that your partner is planning to propose, you can take two approaches.\\\", \\\"Waiting for a truly spontaneous proposal to come to fruition is unheard of. [substeps] Ask your partner what they'd want if you were the lucky one and they were your best friend.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}