{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"75f8eefc-7ef7-5438-8361-4c5198cacebc","task_key":"default--validation--10318","task_revision_id":"1","upstream_id":"10318","short_description":"HellaSwag validation 10318","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to prepare for a wedding [title] Consider hiring a wedding planner. [step] If you're expecting to have a large budget for your wedding, consider hiring a full-time wedding planner. Your wedding planner will help you make a budget, book vendors, and keep you on schedule.\",\"endings\":\"['They can also help you know when to take on new tasks. [substeps] If the exact budget for your wedding is too much, you may want to hire a wedding planner who is a bit more involved in bookkeeping.', \\\"[substeps] If you're worried about budget concerns, consider hiring a part-time coordinator to help with wedding details or a day-of coordinator to help at the wedding. If you don't want to hire a planner, find a comprehensive wedding checklist online to help you stay organized.\\\", \\\"[substeps] If you're in college, you may qualify for a full-time wedding planner. However, you'll likely need someone more experienced in weddings to easily handle your finances.\\\", '[title] Assess your wedding day to see how things can be arranged and how they go. [step] Is there a venue that will suit your budget? Will you be renting small spaces or inviting people over? [title] Get a wedding planner to help you plan.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}