{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2cf7fb18-8b67-57ad-9918-a0cf9c217d01","task_key":"default--validation--43663","task_revision_id":"1","upstream_id":"43663","short_description":"HellaSwag validation 43663","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to plan your wedding music [title] Sit down together and talk about what music matters to you. [step] Each of you will have different ideas and preferences and this is about getting the main ideas heard, to allow both of you to plan in full knowledge of what matters to one another. [substeps] Write down a list of the songs and/or music that you would like to form a part of your wedding.\",\"endings\":\"['Make sure to decide ahead of time the location where you want to arrange your wedding music. You can discuss how many romantic themes you hope to include in each song and set up time for each venue.', 'Ask your future spouse to do the same. Bring the lists along to your discussion together.', 'Brainstorm a logical order together with your partner. Everyone should have their own wants and preferences and think about what exactly matters to them, in order to get included.', 'Or you can look through the music previously choreographed and combine the music together to put into song form. [title] Decide how the music matters to you.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}