{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"80c8345a-0a66-5c04-ba55-4e7ee3d0a9d2","task_key":"default--validation--36029","task_revision_id":"1","upstream_id":"36029","short_description":"HellaSwag validation 36029","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to dress as a winter wedding guest [title] Consider the dress code. [step] Before deciding how the season will affect your clothing choices, you should make sure that you understand the wedding's dress code. Consult the invitation, which will usually make it clear how guests are expected to dress.\",\"endings\":\"['[substeps] For example, in autumn, the dress code may be that guests are not provided with hoods. In summer, the dress code is that guests need to wear uniform scarves, as dress is called for for many seasons in the month of june.', \\\"Only you and the bride will know the dress code and how to behave. [substeps] If your wedding is held at home, keep in mind that spring and summer dresses aren't required.\\\", 'That way, you won\\\\'t dress too formally or casually for the event. [substeps] If the invitation says \\\" white tie \\\", the wedding has an extremely formal dress code.', \\\"This is especially important if a star is getting married in a place of political influence. [title] Make sure it's at least 7 people.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}