{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d4384bcb-71cd-5776-bc48-6fb1b876e086","task_key":"default--validation--38381","task_revision_id":"1","upstream_id":"38381","short_description":"HellaSwag validation 38381","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to dress for a wedding [title] Read the invitation carefully. [step] Often times the invitation will give you direct information regarding what to wear and the level of formality. The style or decoration of the invitation is also a clue.\",\"endings\":\"['[substeps] If the restaurant is casual or super formal, the invitation might tell you to wear a suit. If the restaurant is more casual, the invitation may alert you to how to dress.', \\\"If this is the case, it may be better to wear dress pants, a button down shirt, or long sleeved shirt. [substeps] Even if your invitation gives you direct instructions about specific attire, it's still wise to follow along! You don't want to be embarrassed or to ask someone else for advice you disagree with.\\\", \\\"Use it to find out what the occasion is and make your trip enjoyable by dressing nicely. [substeps] As a guest, it's vital to dress in the same uniform throughout your wedding.\\\", 'Is it colorful and fun or formal and sleek? [substeps] They may mention hints about comfortable footwear by telling you if the ceremony will be on a beach or a lawn. Many people are also using websites to give more information about the ceremony.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}