{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e1f65c01-50bf-5169-b37a-182fe38d4d92","task_key":"default--validation--14395","task_revision_id":"1","upstream_id":"14395","short_description":"HellaSwag validation 14395","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to apply purple highlighter [title] Apply foundation to even out your skin tone. [step] Your skin should be as even as possible when you apply the highlighter, so you should start with a foundation. It's best to use a matte or satin finish formula because the highlighter will add enough glow to your skin.\",\"endings\":\"['[substeps] If you have pale skin, use a matte formulation. If you have skin with a fair complexion, choose a light-colored formula.', 'Use a brush or sponge to apply the foundation for the best coverage. [substeps] For oily skin, choose an oil-free foundation formula to help prevent shine.', \\\"[substeps] If you're simply looking to fill in your eyebrows, draw a thin line across your forehead. The line should start roughly 10 to 15 millimetres (0.9 to 0.4 in) above the eyebrow.\\\", 'Consider adding foundation in whichever direction works best. For example, you could use a powder formula or a bronzer.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}