{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"aa9fb3cd-bf5b-5398-915e-5451f87b044d","task_key":"default--validation--49505","task_revision_id":"1","upstream_id":"49505","short_description":"HellaSwag validation 49505","config":"default","split":"validation","body":"{\"activity_label\":\"Holidays and Traditions\",\"ctx\":\"[header] How to open presents politely [title] Sit down to open presents and open them one at a time. [step] There is no rush to open your gifts, and it seems like you just want what's inside of them if you rush through them. [substeps] Decide which present you'd like to open first, and open the wrapping paper slowly.\",\"endings\":\"['While opening gifts is a nice way to touch the items inside of the gift, it is not the only way you could open the present. Put the \\\" things \\\" inside the wrapped paper and notice what you see, whether you want to find a jewel, necklace, or bracelet.', \\\"Open the wrapper from the box in question instead of packing it up. Once you see your inside the box, don't open the lid yet.\\\", \\\"This will build up suspense, and seem as if it really is a great present once you finally see it. If it's in a bag, pull the tissue paper out slowly and place it beside of you.\\\", \\\"This will allow you to see the small pieces inside, and hopefully give you a little more time to show your cards. If you're not sure what you'd like to open, you can guess instead.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}