{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c65d576d-47e2-5ed5-a4af-18fec1f105f6","task_key":"default--validation--33519","task_revision_id":"1","upstream_id":"33519","short_description":"HellaSwag validation 33519","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to choose a color for suits [title] Make sure having one suit is right for you. [step] All-purpose suits are suits that can be appropriate for nearly any occasion. These suits are great for people that don't need to wear a suit very often and are sometimes called \\\" wedding-funeral \\\" suits because they are meant to be appropriate at either occasion or anywhere between.\",\"endings\":\"['[substeps] This type of suit can work for two different occasions, but may be provided for one not attending either occasion. A simple, and solid color suit with enough depth and some hints of tulle in the back is great.', '[substeps] Some suits might also include a coat or a dress, but these colors usually fall out of those categories. These suits also suit formal parts of formal occasions, i.e.', \\\"Ultimately, these suits are much better than skirts because it's not about appearing feminine or chic. [substeps] A skirt should sit below or above the hips when worn in a funeral and goes to either knee, or this looks unnatural.\\\", '[substeps] These suits need to be versatile so they will be appropriate for most formal occasions. All-purpose suits are designed primarily to be worn to weddings, interviews, and funerals.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}