{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9f747a9d-ae76-53cf-80c9-c0f722a53790","task_key":"default--validation--46931","task_revision_id":"1","upstream_id":"46931","short_description":"HellaSwag validation 46931","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to dress for success [title] Research company culture before you take a meeting or go for an interview. [step] Although there are ways you can dress that are safe, asking an employee or doing a walk-by the main office will tell you if the company expects business formal or business casual. [substeps] In addition, it might also tell you if the employees are stylish, wear color, or don dark colors.\",\"endings\":\"[\\\"[title] Err on the side of formality. [step] If you can't find out what a company or client prefers, wear a suit, polished shoes and classic accessories.\\\", \\\"While this can be nerve-wracking, you can treat yourself to a nice look just about anywhere. [title] Don't let expectations rule you, or your self-confidence will suffer.\\\", \\\"It's important to find out before you do anything. If you dislike the office culture, consider asking another employee whether they would find wearing comfortable clothing enjoyable, but you'll quickly uncover the true answers.\\\", '[title] Bring along an outfit at the meeting. [step] Remember to bring a coat, case or garment bag, on your way out.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}