{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"dee0c2f8-92f9-5160-89bf-40a5eda69ed7","task_key":"default--validation--42216","task_revision_id":"1","upstream_id":"42216","short_description":"HellaSwag validation 42216","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to be a laid back bride [title] Remember that your wedding is not the only event in the world. [step] Talking about your big day excessively will lead to anxiety and most likely annoy everyone around you. If you can limit how much you gush and obsess you and your friends and family will be much happier.\",\"endings\":\"['[substeps] Ask friends and family about what is going on in their lives. Step away from the wedding by getting involved or at least asking about what friends and family members are doing.', 'If you have friends from all over the world, you can name a couple of your most loyal friends. Make sure these people can experience the exciting wedding before you! [title] Mingle with important people.', 'Even if you come home at the end of the night feeling like you are the center of attention, your wedding day is going to be extremely different. [substeps] You can do this by focusing on the beauty in your wedding.', '[substeps] Take time to do the things that you want to do, just as you always wanted to do. Wearing colorful dresses, shining your shoes, and bringing your cute little girl back to school will increase your excitement about the event.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}