{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"98f491f9-4227-5f72-be82-f21341b53611","task_key":"default--validation--172","task_revision_id":"1","upstream_id":"172","short_description":"HellaSwag validation 172","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to look and feel good naked (for guys ) [title] Get naked. [step] That means take off everything. The only way you are going to be truly comfortable facing down your doubts and concerns about being naked is to face them head-on.\",\"endings\":\"['Put on a comfortable, plain t-shirt, some sweatpants, leggings, and a pair of comfortable shoes. Let your thoughts and emotions flow, allowing you to do what you want.', \\\"You don't have to be in stockings and you can never have a capri pants, pair of shorts, or bikini bottoms you deem unsexy. [substeps] You have to be confident, so you have to believe that you are clean-and safe! [title] Keep your underwear on (but not loose! [step] ).\\\", \\\"[substeps] It shouldn't be with other people, as the first step is making sure that you are comfortable being naked. Make sure you are somewhere with a mirror that will let you examine your whole body, especially those parts you can't see directly.\\\", 'Sit down, put on your socks, and pull your knees up. Place your hands on your hips, and stay there.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}