{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"dd7ed552-79c7-5e4f-a373-e12a0ca7eee2","task_key":"default--validation--35163","task_revision_id":"1","upstream_id":"35163","short_description":"HellaSwag validation 35163","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to officiate at a nondenominational funeral service [title] Talk to the family. [step] Call the family and arrange to meet them, preferably in their home. Meeting them in public is ok in a bind, but if that's the arrangement, try not to meet more than two to three people.\",\"endings\":\"[\\\"[substeps] If the ceremony is inside a church, don't bother sitting outside on a park bench. If you're a minister or midwife, it's best to arrive by the usual time of worship.\\\", \\\"[substeps] Of course, only with a family member and one of their kids is okay. If no one at your funeral is willing to attend, that's fine.\\\", \\\"People aren't always happy with the small acts of an officiant. In some cases, the family can become drunk, agitated, or violent during or around a service.\\\", 'If a funeral home is taking care of other arrangements, the funeral directors may have space at the funeral home where you could meet. Invite the family to have other family members and close friends present if they wish, but be sensitive to their needs and to the space considerations.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}