{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a708505b-f7e7-57a4-b40a-65556af8417e","task_key":"default--validation--23893","task_revision_id":"1","upstream_id":"23893","short_description":"HellaSwag validation 23893","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to wear blue lipstick [title] Opt for navy. [step] Navy blue lipstick is flattering on almost everyone. It is also the most popular and easy to find color.\",\"endings\":\"['It is best picked for a feminine look and is fresh and bright with aqua accents. [substeps] Navy lipstick is also the most popular and easy to find color.', \\\"If you don't want to go all out, then go for a brighter shade. You can even always choose lipstick with a hint of bright red.\\\", '[substeps] Choose a soft, neutral color, like light blue or taupe. Avoid bright colors like red, orange, and green as these go against the creamy body and choices.', 'If you are new to blue lipstick, then a navy shade might be the best place to start. [substeps] Navy lipsticks are usually matte, but you can apply a clear gloss over the lipstick if you prefer a shiny lip look.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}