{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4017181d-1e6a-553c-b387-42d62fbc2c74","task_key":"default--validation--36166","task_revision_id":"1","upstream_id":"36166","short_description":"HellaSwag validation 36166","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to relax using the feather method [title] Lie on your back and close your eyes. [title] Rest your hands on your belly. [step] You can also lay your hands on the floor, several inches from your body, with your palms facing up.\",\"endings\":\"['Feet should be spread about shoulder-width apart, or whatever is most comfortable. [title] Take a moment to get comfortable so that you can lie still.', '[title] Bring your arms over your head and apply pressure gently to your buttocks using your palms. [title] Bring your legs up towards your head and pull them to your chest.', '[title] Spread your hands as wide as you can and spread your fingers wide. [step] You can use only your thumb and index or middle fingers.', 'Remember that belly breathing can soothe you! [title] Breathe deeply. [step] For some people, breathing can reduce anxiety and stress.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}