{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a973be46-7e40-557e-9140-fb9a698b662d","task_key":"default--validation--47146","task_revision_id":"1","upstream_id":"47146","short_description":"HellaSwag validation 47146","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to win your wife back [title] Ask her how you can win her back. [step] This may seem too simple or too direct, but it may actually be the most important step to take. Ask her how she feels about what needs to change about your relationship together.\",\"endings\":\"[\\\"Simply by asking, you'll indicate that you care about what she's thinking and that you're ready and willing to put effort into your marriage. [substeps] Be specific with your questions, and ask for specific answers.\\\", '[substeps] Think about all the qualities that have led to her behaving the way she did in the first place, then tell her why. For example, she might have been feeling very vulnerable for a while when you broke up, but perhaps she has insecurities that cause her to behave strangely.', \\\"Maybe you just got back together but aren't able to see eye to eye anymore. This might be a good game to use to figure out how to win.\\\", 'Her answer may be something like \\\" i still love you, and so do you. \\\" is that something you care about? A serious, committed relationship has to happen in order to accomplish what you need to do.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}