{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7c56fc2d-7ccb-5a2e-9df2-66519e4a07c2","task_key":"default--validation--15578","task_revision_id":"1","upstream_id":"15578","short_description":"HellaSwag validation 15578","config":"default","split":"validation","body":"{\"activity_label\":\"Travel\",\"ctx\":\"[header] How to contact kayak customer service [title] Call customer service directly at 1-978-451-0771 or 1-855-529-2501. [step] Depending on where you are located, and your phone plan, you may be charged additional fees for the call. [title] Speak to a customer service representative.\",\"endings\":\"['[step] If this person works at another company, they may be able to identify you on a personal level. Simply use their id number to report the contact to kayak.', \\\"[step] You'll need to speak to them directly or on the phone. [substeps] If you use the phone system for a landline, have the representative direct you to the area code where kayak customer service sells their products.\\\", '[step] Inform them that you plan to call kayak customer service. Remember to start from scratch and be clear about your expectations to reach kayak customer service.', '[step] You may need to wait on the line to speak to a representative about your concern or complaint. Use the wait time to prepare your feedback so you are ready when the representative comes on the line.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}