{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2ad1ae19-3401-58f6-9acd-55a9337b6e81","task_key":"default--validation--7065","task_revision_id":"1","upstream_id":"7065","short_description":"HellaSwag validation 7065","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to reduce funeral expenses [title] Determine if you want a traditional burial. [step] Perhaps the first step you need to take is to research the cost of a traditional burial. Traditional, casket-based, burials are the way that most americans choose to be buried.\",\"endings\":\"['When deciding on a formal funeral, do some research so that you have a good idea of the costs of a traditional funeral. [substeps] Traditional burial can take place at any funeral home where you live.', 'Remember that : [substeps] This option will usually be the most expensive. Costs can and often do exceed $10 , 000, however, depending on what you pick, simple traditional burials can be done for less than $4 , 000.', 'However, the more traditional, the better. Time is the number one factor to consider.', \\\"Traditionally, a traditional funeral will cost around $2 , 000 usd. However, many things, such as plants, crops, and animals, do not require a traditional grave, but you should think about making one if the people living in your home don't qualify.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}