{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"798a48c5-42ee-5671-8c22-0b5cc7f35886","task_key":"default--validation--16073","task_revision_id":"1","upstream_id":"16073","short_description":"HellaSwag validation 16073","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to backboard a spinal injury victim in the water [title] Activate the emergency action plan (eap). [step] Let others know about the situation so they can assist in the rescue. [substeps] Blow your whistle and clear the pool.\",\"endings\":\"['Have another lifeguard or a person nearby call 911. Have another lifeguard or person grab the automatic external defibrillator (aed) and bring it to you.', \\\"You will need emergency personnel as soon as possible to perform a backboard, especially if you need to use the tow rope to extract yourself. If you're scared that you are going to tip over into the water, tell that person ahead of time.\\\", 'Later, have everyone board and the victim sit out and watch the water throughout the day. Be mindful of where the water goes during the night.', 'If the injury occurs in the pool, then try to make sure your passengers cannot see you. [title] Soften the kayak by removing the fuel tank.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}