{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"aae4037f-522d-5566-bd09-ac7e141cdf3c","task_key":"default--validation--2211","task_revision_id":"1","upstream_id":"2211","short_description":"HellaSwag validation 2211","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to wear a trench coat [title] Choose a long trench coat if you're tall. [step] Trench coats come in several different lengths, but this is a common length. Long trench coats can come down past the knee or even just above the ankle.\",\"endings\":\"[\\\"Long trench coats are ideal for taller people, but they may make short people appear even shorter. [substeps] Wear shoes with a heel if you're a shorter person wearing a long trench coat.\\\", \\\"Select a coat that reaches just below the knees to help accentuate your figure. If you're tall, look for trench coats that reach the shins, or underarms.\\\", 'Trench coats are long for coats made by men so you want to choose one with a solid material. [substeps] Trench coats come in different lengths, such as a waist length trench coat, a neck length trench coat, or a shoulder length trench coat.', \\\"Several styles of trench coats have ties that can be worn flat to look outfit-appropriate, or in a wide variety of styles, depending on which pant line you choose. [substeps] Men's trench coats are usually held together with ties that are assigned to men.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}