{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8025df38-831c-53cb-b17d-bd71de80df49","task_key":"default--validation--25753","task_revision_id":"1","upstream_id":"25753","short_description":"HellaSwag validation 25753","config":"default","split":"validation","body":"{\"activity_label\":\"Sports and Fitness\",\"ctx\":\"[header] How to plan a corporate event [title] Conceptualize your big idea, a.k.a. [step] Come up with your theme. [substeps] Whether you are doing a cocktail party, a workshop or the company picnic, setting a theme gives the tone of the event and gives you a starting point when picking your food, your music, your décor and more.\",\"endings\":\"['Consider why everyone in the scene will be attending. Is it being a party to reconnect with family or peers, or is it a plan for just the two? [title] Join the fundraising/organizing committee for the project.', 'It also draws attention to your company goals and the values you have set at the outset. Use these key aspects in choosing your invitations.', 'Use all three to make up your party. Give your guest tables wine and liquor so you never need to add a centerpiece or cake to fill them.', 'Make sure that your theme is appropriate to the crowd for which you are planning. You can find a large variety of theme ideas online to get you started.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}