{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6f040091-a995-59c5-9b6b-e87b7fd3c6fb","task_key":"default--validation--27734","task_revision_id":"1","upstream_id":"27734","short_description":"HellaSwag validation 27734","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to wear brogues [title] Go with full brogue shoes for a detailed, fancy footwear option. [step] Full brogues have a pointed toe cap and extending lines or \\\" wingtips \\\" that run along both sides of the shoe. They have perforations in the leather across the edges, and many styles also have decorative holes on the center of the toe cap (called the \\\" medallion \\\").\",\"endings\":\"['[substeps] From above, the cap shape looks like a \\\" w, \\\" which is why they are also called wingtips. Full brogues are best styled with professional outfits.', '[substeps] When wearing colored brogues, color-issued shoes are often added as a wide-brimmed hat because the toe cap is a shoulder-cap. [title] Wear bright toe pads for a casual look.', 'The plastic ligaments on the inner sides of the shoes are tucked into the toe caps to ensure your toe cap will fit right in. [title] Choose crop tops or cotton-lined dresses for formal styles.', 'These do not carry as much wear on the one end of the shoe. Still, a good footwear of black or brown is important.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}