{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8e117b9d-8a1f-5e5b-905d-3bad76dce03d","task_key":"default--validation--38927","task_revision_id":"1","upstream_id":"38927","short_description":"HellaSwag validation 38927","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to do light daytime makeup and casual hair [title] Create a good base. [step] To make your skin look as blemish-free as possible, start by applying facial primer evenly across your face. Next, put on your preferred foundation and blend it in.\",\"endings\":\"['Another good base color will be a light green for you mom or a dark green for you dad. [substeps] Your base foundation may be a bit drying, and it can be creamy or frumpy depending on your personal makeup preferences.', 'Roll the foundation across the entire face, from the forehead and down across the nose. This will hide the side of your face from the others because they will blend in.', 'Apply concealer under your eyes and on top of any blemishes. [substeps] You can swap out the primer for facial lotion or a light moisturizer.', '[title] Fill out your brows with concealer and white eye shadow. [step] Start at the brow bone and very softly lift at the sides.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}