{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2523096a-0e00-52e8-92eb-b763f29796fc","task_key":"default--validation--21295","task_revision_id":"1","upstream_id":"21295","short_description":"HellaSwag validation 21295","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to be a stay at home dad [title] Do some soul-searching. [step] Make sure you're okay with your wife being the breadwinner, and discuss what you will be expected to do. Are you signing on to do all the housework, cooking, and childcare? Or are you just agreeing to take care of the kids? [title] Consider your career.\",\"endings\":\"['[step] What else do you like doing? [substeps] Do you have a dream job or job? Then you will not necessarily be fighting over child custody. However, think long and hard about what kind of job you plan on taking.', '[step] What can you be doing for what will be most useful to you in life? [substeps] Check out forums and career blogs like aol.com and yelp. You can find a ton of interesting information, such as jobs and job leads.', '[step] Think about how you might balance being a stay-at-home dad with your career goals. Consider asking your employer if you could work part-time and/or from home.', '[step] Spend some time considering what you want to do and how it will affect your lifestyle if your spouse moves. Your spouse might not be exactly ready for that career, but you will be supportive and can give it along.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}