{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1abaec2e-4ccf-5d87-8435-7b14f89a0d3a","task_key":"default--validation--29551--variant-1458c947c549","task_revision_id":"1","upstream_id":"29551--variant-1458c947c549","short_description":"HellaSwag validation 29551","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to pay for funeral expenses [title] Understand the cost. [step] The average funeral costs $7 , 755 according to the national funeral directors association. For some this is not much but for others it is a number they cannot hope to reach.\",\"endings\":\"['[title] Go to the website of the funeral home or crematorium that works in your area. [step] They will give you a telephone number for talking to other funeral directors.', '[title] Know how much money you can afford. [step] Because there are hundreds of different funeral homes in the united states, there are unlimited opportunities to fill all the gaps.', 'This number is also a pay upfront proposition, as funeral homes will no longer accept payment plans. So when you are thinking about planning ahead, know that this is the number you need to target.', 'Contact the people who provided you with the money and ask how much is left over and how much is going towards the death of the person. Also, if the person did not call or email at the time of the death, ask to speak to a funeral director or visit a funeral home.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}