{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e61c2637-1778-5bb5-af5d-a3f00854ad35","task_key":"default--validation--32492","task_revision_id":"1","upstream_id":"32492","short_description":"HellaSwag validation 32492","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to do doll like makeup [title] Apply foundation. [step] Use your finger, a blending sponge, or a brush to apply liquid foundation evenly all over the skin of your face in light, circular dabbing motions. [substeps] Use foundation that matches your skin tone, or is just one shade lighter.\",\"endings\":\"['[title] Apply one layer of eyeshadow. [step] Use a large makeup brush to apply one layer of eyeshadow.', '[title] Blend it in with a makeup sponge. [step] Use a blending sponge to blend off the excess foundation.', 'Allow foundation to dry before applying the fake makeup. Apply blush to your cheekbones, ears, nose and mouth.', 'Feel free to apply foundation makeup a little thicker than you would normally, as the doll look features a porcelain or plastic face that is unlike natural skin. Add concealer to any blemishes, red patches, or dark circles under the eyes so that your face color is completely even.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}