{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"13fd4362-fca5-53f7-8f50-ee98a0e63389","task_key":"default--validation--42379","task_revision_id":"1","upstream_id":"42379","short_description":"HellaSwag validation 42379","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to plan a christmas wedding [title] Look into alternative venues to save money. [step] As many events occur around the holidays, it may be pricier to book a traditional venue like a church. If money is an issue, look into alternative venues.\",\"endings\":\"[\\\"[substeps] If you're having a winter wedding in a warmer region, consider marrying in an outdoor park. Usually, the only fee required is a small registration fee.\\\", 'Either stay at a rural or close-knit venue to save money, or book the venue where you usually do your fancy wedding activities. Before booking your venue, talk to local and national leaders to find out what options are available.', 'You can either opt to have a renaissance wedding in a park where everyone will be happy to go for an afternoon, or a garden with a wide array of flowers. [substeps] Another option is to have a simple church wedding where people in the church and the cathedral go out shopping for food and edibles.', '[substeps] Attending a church service, even if it\\\\'s not a traditional christmas wedding, can save you money. Also, it may be cheaper because you\\\\'ll make a special \\\" guest of honor \\\" as well.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}