{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b3dc7906-c7ce-52c7-af73-e63a63d450e8","task_key":"default--validation--17858","task_revision_id":"1","upstream_id":"17858","short_description":"HellaSwag validation 17858","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to deal with a friend's death [title] Attend the memorial to say \\\" goodbye. [step] \\\" participating in ceremonies like funerals, wakes, or shivas help the living say their goodbyes to the deceased. If your friend has a memorial service, try to attend.\",\"endings\":\"['If you call or go to the service, you may be able to get the whole family to participate. [substeps] Each year, people who attend crematory services will make a donation to pay for physical care to the deceased.', \\\"Many people like to sit around reflecting on the death and thinking about what happened. If your friend doesn't like the service, you may invite her along.\\\", 'You might also bring along flowers to place on the casket, or check with the family to see if you can leave a memento in the casket. [substeps] Sit near their family and/or other friends.', 'Do you really have time? If so, invite your friend to the service with you. [substeps] If your friend dies by accident, empathize with the death, and offer to help the rest of the family.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}