{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b0d90c84-1871-55ee-9a60-94dcaff103b4","task_key":"default--validation--48982","task_revision_id":"1","upstream_id":"48982","short_description":"HellaSwag validation 48982","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to do metallic french tips [title] Remove nail polish and clean your nails. [step] If you are wearing nail polish, make sure you first remove it completely. Take a cotton ball or pad and apply a small amount of nail polish remover.\",\"endings\":\"['Press the pad onto your nail for a few seconds, then wipe off the polish until it is complete gone. Make sure to clean under your nails, as well.', 'Then, wipe your hands and nails down thoroughly with a clean rag. Apply the same to both of your nails.', 'Using this cloth, scrub your nails with the alcohol and polish remover solution. Once you have scrubbed your nails, rinse them with warm water and dry them with a clean towel.', \\\"Using a finger brush, gently move it over your nails like you're peeling off fine old paint. [substeps] Hold the cotton ball or pad between your thumb and your forefinger and carefully work the polish onto your nails.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}