{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c6e94e93-d307-5b96-b7cc-aef9e03918d7","task_key":"default--validation--28559","task_revision_id":"1","upstream_id":"28559","short_description":"HellaSwag validation 28559","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to be a godparent [title] Find out what expectations the parents have. [step] The role of a godparent can vary widely depending on the family. For some people, it might entail attending a child's baptism and confirmation, while other people might expect continual spiritual guidance.\",\"endings\":\"[\\\"Some people may feel the need to attend, although following the parents' faith and chosen path does not seem like a good idea. [substeps] Some people may find the objective to be to communicate closely with the child.\\\", \\\"Generally, you should give yourself time to reflect before you decide your role. [substeps] Some parents suggest attending a child's baptism early in the year.\\\", \\\"Make sure you are clear on exactly what you are agreeing to so that you avoid confusion later down the road. [substeps] Some people even choose secular godparents to help their child's moral development.\\\", 'In some families, you may be expected to perform more biblical duties. If you are not religious, it might also mean attending a church service, or attending an assembly, and so on.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}