{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f8aa994e-c6cd-5948-a4d9-28d53b0f487b","task_key":"default--validation--3569","task_revision_id":"1","upstream_id":"3569","short_description":"HellaSwag validation 3569","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to take care of your grandchildren on summer vacation [title] Hold an exploratory meeting with their parents. [step] Invite them to dinner to discuss the options. Consider your budgets and the grandchildren's interests and ages.\",\"endings\":\"['Bring up the plan (even if it\\\\'s just \\\" if you\\\\'re old enough, \\\" like \\\" in the next year \\\") and ask how they think skiing lessons are appropriate. [substeps] If your grandchildren have a certain percentage of their budget allotted to ski lessons, ease them into it a little more by welcoming them to family activities.', 'Also discuss recommendations for where and how you care for them. [substeps] Watch out for retirement or non-retirement age grandparents (aged 1 to 2 years old).', 'Explain that you intend to travel after the kids fall asleep. Discuss accommodation, and the backpacking possibilities within your days schedule.', '( if you live far away, get everyone on a conference call or use skype. ) budgetary concerns to address include : [substeps] The amount of money the parents can contribute towards expenses.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}