{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3941b70d-b83b-5b46-a8a5-1dbc9151667f","task_key":"default--validation--23656","task_revision_id":"1","upstream_id":"23656","short_description":"HellaSwag validation 23656","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to dress for a rehearsal dinner [title] Check the invitation for dress details. [step] The bride and groom or the groom's parents will have sent you an invitation noting the details of the rehearsal dinner. In addition to the time and date of the event, some invitations note the style of dress the hosts wish for guests to wear.\",\"endings\":\"['Here are some of the different styles of dress you may see listed on a rehearsal dinner invitation : [substeps] Black tie. This style is more common for the actual wedding day, but you may see a request for black tie attire on a rehearsal dinner invitation.', '[substeps] For example, you could check an invitation asking guests to wear elegant suits or dresses with empire necklines. Sometimes, a tuxedo provides a casual look without the emphasis of formal attire.', 'These invitations will indicate their attire so you can tailor your attire accordingly. [title] Get the right attire to attend the rehearsal dinner.', 'In general, most hosts will want to show you these details. From this information, you should consider deciding what type of attire you would like to wear for a rehearsal dinner.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}