{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5d62940d-cee1-57b5-ae66-63801f8ca7cb","task_key":"default--validation--15215","task_revision_id":"1","upstream_id":"15215","short_description":"HellaSwag validation 15215","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make aftershave [title] Choose your main astringent. [step] An astringent is a substance that shrinks body tissues-in this case, your pores. When they shave, most people use warm water, which opens up the pores.\",\"endings\":\"['Take your astringent and ask an attendant at a health store where you can pick it up in liquid form. [substeps] Astringents in-store are slightly more expensive than anything you can buy online.', 'You might consider : [substeps] Rubbing some witch hazel or aloe vera oil onto affected areas. Massaging some astringents on blocked pores.', 'Using an astringent in your aftershave is important because it reverses that process. Astringent also sterilizes any cuts or nicks you accidentally give yourself while shaving.', 'You can use either honey or, in some cases, a reverse astringent, such as citrine. [substeps] Honey tends to help rid your skin and hair of oil.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}