{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"03903294-4343-52dc-853b-fbbd5161ef2f","task_key":"default--validation--17732","task_revision_id":"1","upstream_id":"17732","short_description":"HellaSwag validation 17732","config":"default","split":"validation","body":"{\"activity_label\":\"Sports and Fitness\",\"ctx\":\"[header] How to get a yellow belt in gkr karate [title] Know the process. [step] When you start karate you are a 10th kyu white belt. This means you are a students who is 10 grades away from teaching a black belt.\",\"endings\":\"[\\\"If you start the skater start, you can't learn a new style or use techniques. [title] Learn both the black belt and the flag (which are the two most important rule for str.\\\", \\\"You don't have to wear a karate uniform at this stage, but if you do you wear a plain white belt. This way the sensei's can see that you are quite new to karate and might need some extra help.\\\", 'Without this, the seminars on the topics of strikes, punches, kick and evasion techniques would be out the door. Trying to get a yellow belt in shinbok darko (a style class designed for speed and strength) will be impossible and the weaknesses of this class will be an obstacle.', \\\"You have to choose the university of district you want to attend so as to avoid attention of your robe, it is important that you check with your state's civil and guard status to see if they have any laws regarding the system. As well as the kinds of red belts that become available, based on different situations or roles, all initiates require a master's degree in karate.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}