{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"321df128-2f23-58f2-ae4e-bdd03e600d0b","task_key":"default--validation--49146","task_revision_id":"1","upstream_id":"49146","short_description":"HellaSwag validation 49146","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to write condolence letters [title] Don't wait too long. [step] If you wish to send someone a condolence letter be sure to do it in a timely manner. You should write the bereaved a letter within two weeks of their loss.\",\"endings\":\"['This is the time that they are most focused on grieving. They will be dealing with funeral arrangements and preparing for their final goodbyes.', \\\"Do not wait until either person passes away to leave a letter. You will need to send this after a specified day of the year, sooner if you don't have a coffin to bury them in, or after the funeral arrangements have been taken place.\\\", 'This allows for plenty of time to express your feelings if they have an issue with you about sending them a letter. [substeps] Giving it too long also causes them to be less inclined to accept the love letter.', 'You should also write a few months after that so they can have time to grieve and recover emotionally. [title] Choose a style for a poem (or prose).']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}