{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8921f339-a22f-57ed-afe2-7a0fdebefd9b","task_key":"default--validation--11608","task_revision_id":"1","upstream_id":"11608","short_description":"HellaSwag validation 11608","config":"default","split":"validation","body":"{\"activity_label\":\"Relationships\",\"ctx\":\"[header] How to be honourable [title] Be the person you say you are. [step] It's easy to be a pleasant person, walking around with a ready smile and a \\\" hello \\\" for everyone you see. But being honourable isn't the same thing as being friendly.\",\"endings\":\"[\\\"Being sir or madam isn't for everyone, but not every person just like you. Being classy in a respectful and accepted way is the last thing someone likes.\\\", \\\"While politeness is a necessary component of being honourable, it's not how you say' hello' or' goodbye', but what you say' goodbye' for. Here are some ways to be honourable : [substeps] Don't send mixed signals.\\\", 'When it comes to honor, it\\\\'s more important to be authentic. Show the world who you really are, even if it come at the expense of your reputation for being \\\" nice.', \\\"If your with a friend or relative, ask them if they would like a coffee after work or if they would like to chat with you. If you want to be honourable with a family member, know that they may think you don't deserve to be treated, respect them, and try to spread goodwill and cheer you up.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}