{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ee7d8184-aa84-58f9-b7f6-20b86d0c0319","task_key":"default--validation--33178","task_revision_id":"1","upstream_id":"33178","short_description":"HellaSwag validation 33178","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to get freckle tattoos [title] Look at reference photos. [step] Check out photos of people who have natural freckles online. Use the photos as a guide when deciding how freckled you would like your look to be.\",\"endings\":\"[\\\"[title] Pick the type of shade to choose. [step] Juvenile freckles have a lot of small dents which you shouldn't squish out.\\\", '[substeps] Also use the photos as a guide to determine how light or dark you would like your freckles to be. Choose a couple different looks to test out.', 'Unlined nails, jagged brows, and freckles should all be considered before you buy these tattoos. [substeps] Your tattoos and freckles should not be too obvious.', \\\"[substeps] Other things to look for include bold, bright colors, dark spots, light freckles, etc. If you find someone who has freckles, and they aren't wearing clothes you typically wouldn't mind avoiding, this person may be a good match for you.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}