{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"36a19b5c-cdb7-5e10-af73-efad01ba164b","task_key":"default--validation--1400","task_revision_id":"1","upstream_id":"1400","short_description":"HellaSwag validation 1400","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to do your makeup like eleanor calder [title] First, you want to apply foundation. [step] Eleanor uses the ysl le teint touche eclat foundation. Apply this in a light coat all over your face.\",\"endings\":\"[\\\"[title] Now, brush on eyeshadow over her eyeshadow if she doesn't have her usual eyeshadow. [step] If you choose to have an eyeshadow, simply apply it to her eyelid as a v.\\\", \\\"This product is pretty expensive (40 pounds if you live in the uk and 50 dollars if you live in the usa). If this product is out of your price range, try using the' wake me up' foundation by rimmel.\\\", \\\"[title] Hold the q under the eyebrow where girly victoria's runway wear to the eyebrow, and then apply products to the area to smell your face. [step] ( this is usually done when you're ready to do makeup.\\\", 'The primer helps prevent liner from smearing the liner on your face. Make sure to apply your foundation with a large, wide brush.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}