{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"98d059e5-a90a-5816-87d5-958295f9c3c0","task_key":"default--validation--47508","task_revision_id":"1","upstream_id":"47508","short_description":"HellaSwag validation 47508","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to pick a wedding theme [title] Brainstorm wedding themes with your partner based on your interests. [step] When planning your wedding, it is important for you and your partner to both agree on the theme. After all, this is your special day! Sit down together and discuss potential themes, like winter wonderland or.\",\"endings\":\"['[substeps] Focus on the theme that your partner loves, to help tie your wedding together. For example, if your partner loves to decorate icicles, it may be a good idea to reserve your wedding around summer wonderland or spring wonderland.', \\\"It's now a good idea to have your ideas in order. [substeps] Give your partner a list of some of your favorite classics, such as history.\\\", 'Make sure you also included something you talked about earlier in planning. Schedule a time to come up with a theme while getting your ideas down.', 'You can write down words that define you as a couple to use as inspiration. This helps you come together to get a better idea of what kind of wedding you want to have.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}