{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cb05f32f-3a44-508a-b74a-9e92db863db7","task_key":"default--validation--5396","task_revision_id":"1","upstream_id":"5396","short_description":"HellaSwag validation 5396","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to make a bridal shower fun [title] Speak to the bride before you begin planning. [step] Traditionally, the bride is not involved with the shower, but if she is hard to please, it will be easier to talk through any worries or ideas she might have at the beginning of the process and avoid confrontations that could ruin the day. [title] Decide if a couple's shower would be more fun.\",\"endings\":\"[\\\"[step] If the couple's wedding is impromptu, a shower would be much better. However, perhaps a more intimate shower might be even more fun.\\\", '[step] You can co-plan with the best man and reduce the formality of the event, leading to a more enjoyable time. [title] Determine the budget for the shower to reduce any financial worries.', \\\"[step] At a wedding, some glitz and glamour might also be involved. Decide if a couple's shower could be more fun if you are deciding who's coming to collect the bouquet.\\\", '[step] If the bride has an emphasis on her health, then one of these shower items may be appropriate. If so, then go on ahead with the shower.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}