{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4d6c215d-1b07-5edc-8569-7ce0906460e1","task_key":"default--validation--3635","task_revision_id":"1","upstream_id":"3635","short_description":"HellaSwag validation 3635","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to choose a bath salt [title] Check the packaging. [step] All bath salts start as sea salt, but if they're too processed, they can lose their beneficial qualities. Look for bath salts in packages that read \\\" natural \\\" or \\\" pure.\",\"endings\":\"['\\\" [substeps] Sea salt may not come in powdered or salt form, but it does come in separate packets. Try looking for packets that advertised \\\" natural \\\" and \\\" pure, \\\" which means they are exactly what they sound like.', '\\\" those are the most likely to have retained their best ingredients. [title] Buy your salts from a reputable source.', '\\\" pure bath salt is usually made from natural ingredients, and can have a nutty tang. [substeps] If you go to a grocery store, choose one made from natural ingredients as well.', '\\\" these options are often much safer than sea salt and are usually sold at low prices. [title] Consider adding b12 salts for quicker sinus relief.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}