{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"115d784e-4587-51db-9bee-a8e6da4d0143","task_key":"default--validation--45203","task_revision_id":"1","upstream_id":"45203","short_description":"HellaSwag validation 45203","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to remove spray tan [title] Try a home remedy of baking soda and lemon. [step] Mix lemon juice with baking soda until it forms a paste. [substeps] In the shower, scrub the paste on to your skin.\",\"endings\":\"['You can use a loofah or other shower sponge to scrub it on. You can also just use your hand, since the baking soda will act as an exfoliant.', 'Leave it on for a couple of minutes before rinsing it off. You will notice the sebum evaporating and your skin will feel refreshed and fresh again.', 'The paste will rub the spray tan away, leaving your skin with only the spray tan. Follow up with conditioner to promote intense hydration.', 'For the best results, place the paste in a clean bottle, crack open the cap, and squirt the soap directly in your spray stains. Leave the paste in the fridge until you are comfortable with the way it looks on your skin.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}