{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b9f0abce-e19f-5bd1-b244-a863451a0fad","task_key":"default--validation--48712","task_revision_id":"1","upstream_id":"48712","short_description":"HellaSwag validation 48712","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to celebrate a one week anniversary [title] Get him/her a simple card. [step] A card is enough to show him/her that you've loved being together for the past seven days and that you'd like to stay together for many more. Get a card with something cute and funny.\",\"endings\":\"[\\\"[substeps] For some girls, including: boyfriend couples, don't send hearts, picture frames, ribbon or posters. It'll look really sad and they'll think you're fake.\\\", 'Whatever you pick should be fun, exciting, and funny enough that he/she will remember it for the rest of his life. There are several good ideas, and some good ones are : [substeps] Stay up late on weekends help him/her with homework try a funny trick.', \\\"This brings out a great deal of excitement, and it's a great way to show that you have thought long-term ahead and are still thinking long-term about each other. [substeps] If he/she wants a card on valentine's day or your wedding day, it's okay to give it instead.\\\", \\\"No need to get too serious since it's only been a week! [substeps] Making a card shows her that she means enough to you that you'd take the time to personalize the card with your own words and feelings. Write a silly poem or song about your relationship.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}