{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"898f7145-b8f3-52db-817b-c605793e4e41","task_key":"default--validation--202","task_revision_id":"1","upstream_id":"202","short_description":"HellaSwag validation 202","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to feed an elderly relative in the hospital [title] Find out if the elderly relative has any dietary restrictions. [step] For example: do they have diabetes? Food allergies? Difficulty chewing certain types of food? Difficulty swallowing? Do they need thickened fluids? You can ask your relative's healthcare worker if you don't know of any dietary restrictions. And of course, you can ask the patient directly when possible.\",\"endings\":\"[\\\"[title] Monitor your elderly relative's snack habits. [step] Avoid serving simple snacks like pieces of cheese or hard candies.\\\", '[substeps] Some people may have medical conditions that affect their glucose level. For example: your obese relative may need calcium, iron, and magnesium deficiencies and may need to eat less dairy.', '[title] Check if the elderly relative still drinks plenty of fluids. [step] What they drink when they are ill is important.', '[title] Decide what food is safe to bring your elderly relative. [step] Find out what he or she likes.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}