{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1bd43ac0-e44c-54e2-9087-fbe5bb901a46","task_key":"default--validation--13139","task_revision_id":"1","upstream_id":"13139","short_description":"HellaSwag validation 13139","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to create a wedding memory book [title] Decide the scope of your wedding memory book. [step] Do you want to just document the wedding-related events (the proposal, dress selection, bridal shower, wedding day, etc. ) or would you like to include memorable events throughout your relationship and engagement? This determines the length and photo selection of your book.\",\"endings\":\"['This map will help you choose a good book. Always make sure that an event you want to have relatable usually falls within the scope of your wedding memory book.', '[title] Look through the proofs of your wedding photos and mark the ones that stand out to you. [step] Go with your gut feeling and err on the side of inclusion--you can sort out the bad ones later.', '[substeps] For example, be sure to include pictures of your wedding events to assign to your book. [title] Write the hopes and dreams of each person in your book.', '[substeps] It may help for you to consider what works best with the image of the bride and groom. The book may reflect certain moods, seasons, and places or experiences which essential elements become important.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}