{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ad9ec955-b205-53f1-bb61-813c2d994be9","task_key":"default--validation--40182--variant-1bb57f8daf15","task_revision_id":"1","upstream_id":"40182--variant-1bb57f8daf15","short_description":"HellaSwag validation 40182","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to look good in shorts [title] Take another person shopping. [step] If you can, always have a shopping buddy. This person can be a friend, family member, or significant other.\",\"endings\":\"['Go shopping with a friend if you can or an acquaintance who can help you out if you find a time to use the bathroom. Talk with your friend about which type of clothes you like.', 'If they have a younger sister or cousin, this will help. It also needs to be someone who understands that you love shopping.', \\\"While this is not necessity, do also try to pick someone you don't know well. You can make them doubt themselves or perhaps feel insecure.\\\", \\\"When you try on different pairs of shorts, ask him or her if you look good in them. [substeps] It's important to get someone else's perspective when shopping, since too often people are their own worst critics.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}