{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e2c19385-6d87-5949-b1dd-72f840c0bd0d","task_key":"default--validation--6339","task_revision_id":"1","upstream_id":"6339","short_description":"HellaSwag validation 6339","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to apply nail wraps [title] Remove any nail polish. [step] It is important to remove any old nail polish before you apply your nail wraps. Use some nail polish remover to take off your old nail polish.\",\"endings\":\"['If it remains, you risk bacteria causing nail polish from reusing it. [substeps] Rub a damp cotton ball with nail polish remover over a small area of your nail.', 'Even if you do not have any nail polish on, you might want to swipe each of your nails with a little nail polish remover to ensure that they are totally clean. [substeps] You can also wash your hands and nails to ensure that they are free of grease and dirt.', 'After you clean your nails, allow them to dry completely before applying your nail wraps. [substeps] Nail polish remover can sting if you do not drink water.', 'This will remove the old polish remover onto your skin. [title] Place the nail wraps onto your skin.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}