{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"05dcfc0b-9a02-52f1-8735-ee6befe0d615","task_key":"default--validation--8753","task_revision_id":"1","upstream_id":"8753","short_description":"HellaSwag validation 8753","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to apply a mud mask [title] Scoop out some beauty mud. [step] Use your middle and ring fingers to scoop out some of the beauty mud. Start with a small amount, about the size of a dime.\",\"endings\":\"['The size of your finger should be fine, as the mud is intended to apply to your complexion. [substeps] Wash your hands and your face before you scoop out the mud.', 'You could decide to use other small amounts depending on how thick you want the mud to be. If you are only doing a smaller amount, your mud is more likely to stick.', \\\"If you'd like to apply a thick layer of beauty mud, you can scoop out a larger amount. [substeps] Remember, it's easier to add beauty mud to your face than remove excess.\\\", 'Most products come with an applicator. Some include face powder or petroleum jelly, but this varies widely in size.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}