{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3b1bbf0e-ffb7-54dd-8b31-85786c31a0e8","task_key":"default--validation--44225","task_revision_id":"1","upstream_id":"44225","short_description":"HellaSwag validation 44225","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to live apart and stay married [title] Take advantage of technology if you're in a long-distance marriage. [step] You or your spouse might have to live in another state or country for work, school, or another obligation. If living apart isn't a choice, schedule regular video and phone chats to keep in touch.\",\"endings\":\"['[substeps] There are websites that host face-to-face conversations in languages both can use to communicate often. There are also paid chat services that can help couples talk without having to share information or use video with their dependent family members.', 'Text messages and video chats help even out the long distance relationships. [title] Set up your own social media accounts.', '[substeps] Skype or email is a great solution. You can even change your phone number as well.', '[substeps] Additionally, try sending each other texts or emails here and there such as, \\\" hey babe, hope you\\\\'re having a great day! Love you. \\\" technology can also help you keep your sex life alive.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}