{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b6c95b22-3dff-50ff-84f5-6131496f3851","task_key":"default--validation--41427","task_revision_id":"1","upstream_id":"41427","short_description":"HellaSwag validation 41427","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to plan your own funeral [title] Make a list of your likes and dislikes. [step] Take out a pen and some paper and begin brainstorming ideas regarding your funeral. Write down whatever comes to mind, at least at first.\",\"endings\":\"['Then, come up with ideas for different funeral traditions (including : [substeps] People from different backgrounds a place to put the ashes for the deceased. Did she like listening to her favorite music or watching a movie? People from different religions maybe.', '[title] Prepare for your funeral by creating a family obituary. [step] By visiting your funeral home to have them draw up a letter informing you what they wish to say, they can write the letter at bedtime.', \\\"You might include things that you've seen done at other funerals that you though worked well. [substeps] As you are sketching out ideas, pay attention to the intangibles as well.\\\", 'It might be something simple like the experience of sitting on the floor in front of a wall clock or getting the whole room to flood with the sound of ocean waves...whatever you think of. [title] Begin planning your funeral as soon as you are aware of it happening.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}