{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b766ba8a-31db-594e-b04c-a1365326e4df","task_key":"default--validation--39120","task_revision_id":"1","upstream_id":"39120","short_description":"HellaSwag validation 39120","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to know where to sit at a wedding [title] Have an usher show you to your seat. [step] The ushers are there to seat guests and control the flow of foot traffic into the venue as people start arriving. Let the usher know whether you're a guest of the bride or the groom.\",\"endings\":\"['Setting up your seating is an important component of planning a wedding in a church wedding. It requires a clear understanding of the aisle.', 'If you are not a guest, make sure the usher allows you to eat, drink, and prepare food before throwing a party. [substeps] The usher keeps the doors open for guests, maintains general cleanliness, and helps you out if your area is prone to staying this way overnight.', \\\"According to tradition, the bride's side is to the left of the aisle when facing the altar, while the groom's side is to the right. [substeps] If you don't have a preference for where to sit, you can fill in an empty seat on either side of the aisle.\\\", \\\"If you're a guest of the groom, you don't have to be seated at their place of your choosing. Simply ask if your chair is available when the usher.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}