{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"04a37eea-baf6-5c73-8b13-3ffc5a02e175","task_key":"default--validation--49949","task_revision_id":"1","upstream_id":"49949","short_description":"HellaSwag validation 49949","config":"default","split":"validation","body":"{\"activity_label\":\"Sports and Fitness\",\"ctx\":\"[header] How to cancel a ymca membership [title] Contact your local ymca about their cancellation policy. [step] Start by calling or emailing your local ymca branch for information on their specific cancellation policy. Look for their contact number on their website or on the back of your ymca membership card.\",\"endings\":\"['You can also visit your local ymca branch in person and speak to a representative at the front desk for more information. [title] Give notice 10 to 30 days in advance.', \\\"You will want to contact ymca directly to finalize the cancellation. Once you have talked to ymca's administrators, they will either tell you the reason why or tell you specifically the special cancellation policy you should comply with.\\\", 'This contact information will give you information regarding which cancellation policy you will need to change. [substeps] You can also find your local ymca short-term membership agent by doing an internet search.', '[substeps] Check with your local ymca about due dates for cancellation; not all ymca branches will allow you to cancel your membership. [title] Report violations of your ymca membership card within 120 days of its cancellation policy.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}