{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2564f9b5-0e20-57ff-a05d-a37d9113db4e","task_key":"default--validation--14134","task_revision_id":"1","upstream_id":"14134","short_description":"HellaSwag validation 14134","config":"default","split":"validation","body":"{\"activity_label\":\"Sports and Fitness\",\"ctx\":\"[header] How to be a successful sports player [title] Be a \\\" go-getter \\\". [step] The first step to be a go-getter: have faith in yourself! A go-getter is a person who can get things done and achieve desired goals, no matter how bad the situation is or how limited the resources are. [title] Contribute.\",\"endings\":\"['[step] One perfect way to achieve success is to contribute to things you are passionate about. This of course means dedicating your time and energy to the cause you care about (like your teammates) and potentially caring for them.', \\\"[step] Don't just sign up just to be on the team. It doesn't matter about your skill, but your effort and willingness to contribute is more important.\\\", '[step] When you contribute, others will want to join in it, and in turn you will help build social status. Never be bad about contributing, but always keep for personal reasons.', '[step] Get involved in a team sport at least once a week, set your own schedule, and ask team members for suggestions when they need you. Provide a compliment or an award if you know what they need.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}