{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0febbf49-b420-5fd7-9d52-9adeeb3c6af3","task_key":"default--validation--12650","task_revision_id":"1","upstream_id":"12650","short_description":"HellaSwag validation 12650","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to draw on your eyebrows [title] Prepare your canvas. [step] Before applying any makeup, your face should be cleaned, moisturized, and dried. If you wear a full face of makeup, make sure the following steps are finished before starting your brows : [substeps] Begin with a layer of makeup primer.\",\"endings\":\"['Go 2-3 shades darker than your natural skin tone to layer your concealer and eye shadow with your foundation. Apply foundation beneath your brows using a brush or sponge.', 'Use a small round makeup brush to mix the primer. Spray a thin layer on both your brows.', 'Apply this first primer on the iris or right brow area, then on the face, above the eye line and under the lid. Apply a layer of opaque liquid concealer on the iris, right under the eye area, followed by your own eyeliner in the same place.', 'Primer will prevent your makeup from smudging and help it to last longer. This is extremely important when trying to maintain the illusion of natural eyebrows on bare skin.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}