{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bfa2d815-eaed-5d29-8d78-f2ac786ccd4e","task_key":"default--validation--6806","task_revision_id":"1","upstream_id":"6806","short_description":"HellaSwag validation 6806","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to wear cowboy boots [title] Embrace the boot. [step] If worn well, cowboy boots can be a fashion statement for both men and women. Why not embrace them? Cowboy boots have a long and storied past: think cowhands, ranchers, john wayne, and clint eastwood.\",\"endings\":\"['Both of these boot styles have a special touch to them. Casual or outfit-fit boots will work well for some occasion.', \\\"Pair the boots with luau-inspired pants, and you'll jump right in there. However, cowboy boots can be something vastly different that compliments your slim body frame.\\\", 'They can be manly for men, stylish for ladies, and a great addition to a wardrobe. [substeps] The height, tapered toes, and decorations make cowboy boots distinctive.', 'Always wear cowboy boots if you want to appear bigger and broader than your intended appearance. In addition, the material should come up to your armpits.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}