{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5ed3132b-55a8-5e40-b93f-300c8086aa2b","task_key":"default--validation--22844","task_revision_id":"1","upstream_id":"22844","short_description":"HellaSwag validation 22844","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to apply simple makeup [title] Wash your face. [step] Prior to applying makeup, you must properly prepare your skin for the application. To remove dirt and oils, thoroughly cleanse your skin with a gentle facial wash product.\",\"endings\":\"['Dab your face dry with a clean towel. [substeps] If there are any traces of makeup remaining, scrub your face in small circles with a cotton ball or swab dipped in makeup remover.', 'The washing product should be free of pore-forming oils. [substeps] For oily skin, consider using \\\" murphy\\\\'s oil \\\" or \\\" neutrogena oil \\\" to flush your skin in between washes.', \\\"It may take up to thirty minutes for the ingredients to penetrate your skin. [substeps] As these products contain metallic compounds, it's important that they soak into your skin before you apply makeup to cover the chemical grime that they cause.\\\", '[substeps] You can use any face wash you like: plain water, filtered water, or a combination of all of these options. If you are not particularly worried about putting on makeup, this option is usually not recommended for skin looking dirty.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}