{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ca469da8-5e87-570d-b50a-c4eb29dc2797","task_key":"default--validation--14440","task_revision_id":"1","upstream_id":"14440","short_description":"HellaSwag validation 14440","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to get shampoo out of your eyes [title] Calm down. [step] When shampoo is in your eyes, you might experience a burning or stinging sensation. Pain can often lead us to panic.\",\"endings\":\"['The exact cause of this burning is unknown, but because of this, it is important to calm yourself down. [substeps] This is important because pain is often a response to unavoidable stimuli.', 'Take some time to calm down and prepare yourself for the coming event. [substeps] Try to remember that any kind of extreme stress or anxiety will trigger a panic attack.', \\\"Staying calm will ensure you do not do anything which might make the situation worse. There are few options for calming down when you're confined to your shower, but one way to calm down in such a situation is to take control of your breathing.\\\", 'Find someone who can help you calm down by being able to speak clearly. Sometimes it is comforting to have someone speaking through you, like a family member or friend.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}