{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d6fac853-7184-5587-b615-fca4250e9351","task_key":"default--validation--30316","task_revision_id":"1","upstream_id":"30316","short_description":"HellaSwag validation 30316","config":"default","split":"validation","body":"{\"activity_label\":\"Youth\",\"ctx\":\"[header] How to be a good step mom [title] Give your stepchildren time. [step] Women are more likely than men to feel responsible for making their stepfamilies perfect. However, remember that most stepchildren will need time to adjust.\",\"endings\":\"['You may find that you miss being around your friends, jobs, or family members who are not ideal for you or your kids. Serve children as you would grandchildren when they are old enough.', 'Set aside time for children in your care as children will feel less neglected. [substeps] Break your kids out of home as much as possible.', 'Some may be open from the start, while others may be more reluctant to engage. [substeps] Recognize that the transition is hard for them.', 'Do not call, text, and ruin plans with them. Allowing for extra time for your children will make their natural changes much more cathartic than time away from them.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}