{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1cb6d0bb-436f-5d00-b18f-557df45e7a4d","task_key":"default--validation--18232","task_revision_id":"1","upstream_id":"18232","short_description":"HellaSwag validation 18232","config":"default","split":"validation","body":"{\"activity_label\":\"Youth\",\"ctx\":\"[header] How to be proper [title] Be clear on the type of event. [step] Before accepting an invitation or agreeing to tag along somewhere, ask a lot of questions. Find out where the event will take place, what type of event it is (formal or informal) and who will be there.\",\"endings\":\"[\\\"It doesn't matter whether the party you will be attending will be formal or informal. You can still smile and make eye contact when interacting with the locals, but make sure that you are ahead of the game as well as providing a good excuse to split off every so often.\\\", 'Getting a ton of information upfront will allow you to asses the situation before you get there and make the right decisions so you can fit in. [substeps] If the event is formal, for example, a great follow-up question would be regarding the dress code.', \\\"Don't ask these questions unnecessarily. Chances are it's a group event.\\\", 'Have the venue arrange for transportation to and from the event. That way, you can get there quickly if you need to.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}