{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"78eac5c4-3e74-58ea-bec5-106173ec13fc","task_key":"default--validation--23649","task_revision_id":"1","upstream_id":"23649","short_description":"HellaSwag validation 23649","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to grow a handlebar mustache [title] Gather your supplies. [step] First, choose a wax for your hair. There are many kinds of mustache wax available.\",\"endings\":\"['If your mustache hair is thick and wiry you should buy a stiff wax. Inversely, if your mustache hair is thin, you can buy a softer wax.', 'You need to pick a hair type that you like. [substeps] Consider growing mustache hair on a slightly smaller facial section.', \\\"Although these wax aren't as sparkly as your everyday specialty wax, they're still attractive. You may be able to use other wax when you're creating your facial beard.\\\", 'You can go for the shaved version or the more traditional style. Most handlebar mustache wax is made from natural ingredients.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}