{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3095478f-4e3c-5ce7-8cb1-06120be90a96","task_key":"default--validation--44507","task_revision_id":"1","upstream_id":"44507","short_description":"HellaSwag validation 44507","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to perform rescue breathing [title] Check the scene for danger. [step] Your first instinct is to rush into to help someone in trouble, but you don't want to put yourself in danger in the process. Look around the area to make sure it is safe for you to help.\",\"endings\":\"['Ask someone close to you that you can borrow a flashlight for a while or enact rescue breathing if they will give you one. Reassure the person that you are helping and help them feel safe.', '[substeps] For instance, you should look for things live electrical wires, falling rocks, live power tools, or people with weapons. Also, make sure that you and the person in trouble are out of the way of oncoming traffic if you are near a roadway.', 'If there is blood, you can stay and watch for injury. If there is a survivor, turn her over.', 'Do not touch a person without first looking at the danger, and do not approach or move the person until you observe the situation and have an open mind and feel there is nothing in the way. [substeps] If you do not know for sure what to do, it is best to look around.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}