{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d5f3d439-2771-572a-bc0a-d0a669312bf4","task_key":"default--validation--10681","task_revision_id":"1","upstream_id":"10681","short_description":"HellaSwag validation 10681","config":"default","split":"validation","body":"{\"activity_label\":\"Work World\",\"ctx\":\"[header] How to apply for paternity leave [title] Read your employee handbook. [step] See if your company policies include the option of taking paternity leave. Keep in mind that if your employer offers any kind of maternity leave for new mothers, it is required by law to offer comparable paternity leave for new fathers.\",\"endings\":\"['You will have no legal recourse if you are denied paternity leave. [substeps] Consult your handbook for the standard statement or guidelines on paternity leave.', 'Read your employer handbook closely to see if you and your employer have decided on the frequency and duration of a designated paternity leave. This is important, as you should bring this information with you to any applications.', '[substeps] Employers must offer \\\" parental \\\" leave, not \\\" maternity \\\" or \\\" paternity \\\" leave. Benefits that are only available for use by one gender can give rise to discrimination lawsuits.', 'Look for any rules that might be put into place by the employer, including their knowledge of the maternity leave act. [title] Inquire about who wants to have paternity leave.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}