{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"54b25ecb-36bf-5646-8837-882cfee8fd20","task_key":"default--validation--38586","task_revision_id":"1","upstream_id":"38586","short_description":"HellaSwag validation 38586","config":"default","split":"validation","body":"{\"activity_label\":\"Sports and Fitness\",\"ctx\":\"[header] How to compete in shot put events [title] Be sure to let your coach know that you would like to compete in the event. [step] S/he will be able to help you train for it and will be the one signing you up at competitions. [title] Do weightlifting training.\",\"endings\":\"['[step] Doing weights helps you build strong muscles and endurance. Most olympic qualifiers involve doing bodyweight exercises (if needed).', '[substeps] This will be a very important aspect of your shot put training. It will strengthen your muscles to throw farther.', '[step] Muscles need lots of work to have stable, strong body. 8-10 hours of weightlifting every day will work the same muscle groups, including the normal arms, legs, and back.', '[step] Each event will require you to perform certain exercises or push ups in order to improve your technique. Until your coach tells you to do this, do regular body weight exercises without using any weightlifting equipment.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}