{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3ab4784b-08e8-5f53-9b16-306252e55de2","task_key":"default--validation--19977","task_revision_id":"1","upstream_id":"19977","short_description":"HellaSwag validation 19977","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to apply different styles of eyeshadow [title] Apply a sheer colour all over your lid, up to your eyebrow. [title] Softly line your eye with a pencil liner. [step] It should be a dark or medium shade, like black or grey.\",\"endings\":\"['Smudge the liner upwards to create a small line, rather than in a dramatic swoop. [substeps] If you use the pencil, smudge it slightly with your finger before closing your eyes.', \\\"[title] Apply a shade of eyeliner with the darkest eyeshadow you can think of. [step] Use tiny dots to close your eyelid, so the shade doesn't come out behind your eye.\\\", '[title] Focus on the edges of your eyes, blending with the darkest edge. [step] This will make your eyeshadow blend more quickly and take away from the normal depth of intensity.', 'The line should go from the inner corner to the outer corner. Use the same pencil to dot below your your lower lashes.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}