{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"39f7437b-6ec9-5ca2-aff1-7f5f31c3dbbc","task_key":"default--validation--12856","task_revision_id":"1","upstream_id":"12856","short_description":"HellaSwag validation 12856","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to dress for a black tie event [title] Read the invitation closely. [step] There are subtle difference between \\\" black tie optional, \\\" \\\" black tie preferred, \\\" and \\\" black tie. \\\" knowing the time of day and the time of year for the event, as well as what kind of event it is, can help you determine how to dress best.\",\"endings\":\"['[substeps] Generally, black tie attire is minimal. Light colored suits and business shirts are not required.', '[title] Wear something that will put you ahead of other attendees. [step] For example, a leather or dress jacket is a great option.', '[title] Wear houndstooth with a long dark trench coat. [step] This is common attire for black tie events, but it may also suit some different needs.', '[substeps] Evening events are more formal than daytime events. Winter events are often more formal, and dark colors are more appropriate, than summer events.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}