{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3e8fbd5d-b5a9-5edb-abc5-65b8fac26891","task_key":"default--validation--17031","task_revision_id":"1","upstream_id":"17031","short_description":"HellaSwag validation 17031","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to pick music for a funeral [title] Take the venue into account. [step] If the memorial service is going to take place at a funeral home, you will be able to play almost any music that you like. However, if the service will be held at a religious establishment like a church, then you may be held to the traditions and restrictions of that religion.\",\"endings\":\"[\\\"[substeps] Consult the venue's website for information regarding catholic worship practices and service requests in your area. Check the location of the funeral home beforehand.\\\", 'Before choosing any music, however, consider the dynamics of the venue before purchasing music. [title] Check if the venue specifies rules and cleanliness.', \\\"Be sure to consult with a clergy member to discuss any possible rules in regards to music. [substeps] Even if you can't play particular songs during a service, you may be able to play them while people are entering/exiting.\\\", '[title] Take into account the location of the burial location. [step] Bury at an open-casket grave site, or in a bereavement cemetery.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}