{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"33c4ef0f-ba37-5a4d-814e-12f8c963791a","task_key":"default--validation--48423","task_revision_id":"1","upstream_id":"48423","short_description":"HellaSwag validation 48423","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to conduct a memorial service [title] Get help planning the memorial service. [step] The death of a loved one is a difficult time and planning a memorial service can be a daunting task. You don't have to go it alone! Enlist the help of friends and loved ones with the planning and preparations right from the beginning, and consider hiring outside help if needed.\",\"endings\":\"['[substeps] Make sure you have trusted the permission of your friends and family for the memorial service to be appropriate. Talk to any doctors, nurses, and aides who will be attending the memorial service.', '[substeps] In any event, make sure that you contact your local memorial service center early so that the time frame is set. This may seem like too much time, but asking about a final date and time space beforehand can eliminate waiting time.', \\\"Just because an elderly person has died doesn't mean they'll be forgotten. They will still be remembered no matter what you do when the time comes.\\\", '[substeps] Most houses of worship have people on staff who can give guidance in this process. Funeral directors will also help, but will charge for their services.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}