{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"515c58a7-89e2-5646-b58d-0faf47165442","task_key":"default--validation--38968","task_revision_id":"1","upstream_id":"38968","short_description":"HellaSwag validation 38968","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to be classically beautiful [title] Wear classic pieces. [step] Classic women never wear trendy clothes. They wear classic pieces such as a simple black dress with a striking accessory, aviator glasses, a colorful scarf or a pretty set of pearls.\",\"endings\":\"['Elegant and classy pieces make you look self-confident and graceful. [title] Brush and style your hair as you normally would.', \\\"Think of jacqueline kennedy onassis's famous over-sized sunglasses and timeless suits. You don't have to go back to the nineteen-fifties, but expand your wardrobe when possible, and learn to have your own sophisticated style.\\\", \\\"Feel free to experiment with their style, but try to avoid coming across as boring or uninteresting. Buy something that makes you feel like you've made it for yourself, and act casual.\\\", 'Elegant, classy women should choose dresses that are form-fitting, but that are not too tight. Men should aim for dresses that hang to their mid-thigh.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}