{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"30094396-3a76-5ef5-8f64-ffd7cc7f3f96","task_key":"default--validation--4477--variant-f4affd2e3a2b","task_revision_id":"1","upstream_id":"4477--variant-f4affd2e3a2b","short_description":"HellaSwag validation 4477","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to use a lash curler [title] Get a high quality eyelash curler. [step] There are many brands, but some are better quality than others. [substeps] Use a metal curler rather than a plastic or rubber one.\",\"endings\":\"['Plastic gives the illusion of weight and adheres to your skin less effectively. Alternatively, buy a wand from a beauty store.', \\\"A high quality curler will be made of sturdy metal with rubber pads on the clamps. Plastic or rubber curlers don't get a good enough grip to curl eyelashes well.\\\", 'Plastic is better because it provides less friction. The most common point of contact is the outer or middle part of the eyelid.', 'Plastic cubbyheads have an acidic look and liquid detergents can actually leave behind dye that discolors and could leave permanent marks on your skin. [title] Dab the curler with wet saline.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}