{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"98d16543-61b1-53b2-956e-1938390579cf","task_key":"default--validation--27899","task_revision_id":"1","upstream_id":"27899","short_description":"HellaSwag validation 27899","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to shave your neck when growing a beard [title] Visualize a line from one ear to the other. [step] Imagine a line that loops from the back edge of each ear down and around the top of your neck. Envision the lines meeting in the center of your neck (below your jaw).\",\"endings\":\"['[title] Close your eyes and draw a line halfway along the back, below your cheekbones. [step] Be aware that this should be both long and tapered, to simulate a straight neck in this image.', '[title] Visualize two lines extending down from your sideburns. [step] Find the rear edges of your sideburns.', \\\"[substeps] If you are shaving inside or outside of your ear, concentrate on the lines. Don't be surprised if the lines are poorly defined at first; over time, they'll become more defined as you become more proficient at shaving.\\\", 'If you feel the top side of your neck rubbing against this line, you probably need to shave for your beard. [substeps] In your dreams, take the entire scene in your mind and try to create a picture that feels real and realistic.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}