{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"93265118-3661-5bab-adc2-9f23903b9e78","task_key":"default--validation--26807--variant-e1b4148db746","task_revision_id":"1","upstream_id":"26807--variant-e1b4148db746","short_description":"HellaSwag validation 26807","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to apply foil eyeshadow [title] Gather your materials. [step] Before you begin, you should have all your materials ready. You should have all your regular makeup and eyeshadow supplies.\",\"endings\":\"['You may also want to pack a concealer brush and makeup remover. Keep everything else ready to go in your purse and backpack.', 'The prep process will take some time so be sure to start with the preparation first. [substeps] The preparation is to apply eyeliner with a dark eyeshadow brush.', \\\"In addition to this, you'll need some supplies specifically used for foil eyeshadow. [substeps] You will need pigmented eyeshadow in a color of your choice.\\\", 'Your basics will be : [substeps] Eyeshadow palette: an ultra-bright eyeshadow brush small fluffy spoon eyeshadow applicator [title] Select the right color palette. [step] The right palette is the body of the folded eyeshadow.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}