{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"dcfced29-4f41-5f10-9e33-2a41a6db5a3f","task_key":"default--validation--44733","task_revision_id":"1","upstream_id":"44733","short_description":"HellaSwag validation 44733","config":"default","split":"validation","body":"{\"activity_label\":\"Finance and Business\",\"ctx\":\"[header] How to compare drug plans [title] Compare plans every year when the enrollment period opens. [step] Depending on where you live, you may need to wait for a specific time of year to sign up for a plan or make changes to your existing plan. Search online, consult your government's insurance regulations, or speak with an insurance company for more information.\",\"endings\":\"['[substeps] An enrollment period allows at least one month to pass after a year of regular drug administration. Be aware that enrollment periods in the us are completely variable.', 'If you have a recurring drug plan, contact your insurance company to book a pre-licensing evaluation. Check by phone or mail as in an enrollment process.', 'The enrollment period happens at least once a year. [substeps] In the u.s., for example, the main enrollment period spans from october to december.', '[substeps] Many plans only come with a lifetime guarantee and you can have them in five years. [title] Understand the benefits of a \\\" prepaid new car \\\" plan.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}