{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"201d78ae-4431-57f2-84b8-4f10d6360ca0","task_key":"default--validation--25360--variant-0d1c07170b59","task_revision_id":"1","upstream_id":"25360--variant-0d1c07170b59","short_description":"HellaSwag validation 25360","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to calm oneself [title] Breathe. [step] Just breathe. Slowly.\",\"endings\":\"['Focus on your breathing and relax your body. Continue to breathe in and out slowly through your nose, keeping pressure on your diaphragm and gluteus (your lung).', 'If your heart is beating fast or hard, you might be in danger of tachycardia, or stroke, or any other heart attack. [title] Lay on your back and breathe in and out deeply.', 'Breathe in for 5 seconds; hold for 5 seconds and then out for 5 seconds. This works because you are doing the opposite of what an excited person would be doing.', 'Inhale for a count of five and lower yourself back down. Breathe out, exhale four more times, as deep as you can feel, and repeat for a total of ten.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}