{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a651ea35-d336-53ec-b4f7-1635aeff0eef","task_key":"default--validation--23239","task_revision_id":"1","upstream_id":"23239","short_description":"HellaSwag validation 23239","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to dress well as a tall and thin man [title] The suit is a very important part of a man's wardrobe. [step] A suit which has checks, window pane fabrics are best for tall men as the horizontal lines in them seem to decrease the height, while adding broadness to the frame. [title] The best colors for a suit are lighter shades.\",\"endings\":\"[\\\"[step] The best colors for a business suit are light grey, tan, beige, and medium navy blue. [title] The unmatched suit is a great way to cut the vertical lines of the tall man's silhouette.\\\", '[step] Darker tones are preferred, e.g. navy, black, dark gray, dark brown, and dark white.', '[step] Darker shades will stand out in your shirt and pants so choose a shirt that compliments the color of your skin to make your look more attractive and fashionable. Darker shades of clothing paired with dark pants and neutral neutrals will make your chest look slimmer.', '[step] His clean sneakers, blue tie, or black dress shirt should be good. Because it is his, avoid being overdressed; consider wearing different colors or shades when going out for a meeting.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}