{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9a5d0e4a-8528-5fb2-ac2c-260d9015d925","task_key":"default--validation--26911","task_revision_id":"1","upstream_id":"26911","short_description":"HellaSwag validation 26911","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to fix neck pain [title] Lie on your back with \\\" cactus \\\" arms. [step] Lay down on the ground. Keep your feet on the floor with your legs bent, or stretch your legs out straight.\",\"endings\":\"['When you lie, take a long deep breath in through your nose. Hold this for a few seconds, then exhale and lie down on your stomach with your arms up on the floor.', '[title] You will see a slight recovery after a few days of lying on your back. [step] Keep your neck especially protected by the fabric of your clothing.', 'Bend your elbows, keeping your arms on the floor and your palms facing up. Your arms should frame your face, and look kind of like a cactus (or goalposts).', 'Position your knee in the exact place where you think the problem muscle would be. Make sure you are keeping your left elbow against your right thigh.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}