{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fbba347f-1f3e-5847-86ea-c387e86e2794","task_key":"default--validation--8523","task_revision_id":"1","upstream_id":"8523","short_description":"HellaSwag validation 8523","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make your eyelashes and eyebrows look good [title] Determine the perfect eyebrow shape for your face. [step] Grab a straight edge of some sort, like a pencil or a ruler. You'll also need an eyeliner pencil, or anything that you can use to mark up your brows.\",\"endings\":\"['Using these two tools, you can create the ideal shape for your features. [substeps] Hold your straight edge against the side of your nose, so that it runs parallel.', 'The pencil should be fine, thick and heavy. [substeps] Use an eyebrow pencil at least 3-4 shades darker than your natural hair colour.', '[substeps] To determine the shape, sketch a square-like fracture across the cross portion of each brow. Create the alignment at the bottom, going up and down along your eyelid.', \\\"Starting at the top of each eyebrow, trace the bottom edge of your eyebrows and connect them to the top edge of your eyebrows with your pencil. [substeps] For best results, you'll want to do this in the mirror before you do it.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}