{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"def22c3e-b962-514f-a950-dadda6e72be8","task_key":"default--validation--3474","task_revision_id":"1","upstream_id":"3474","short_description":"HellaSwag validation 3474","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to apply magnetic nail polish [title] Clean and dry your fingernails. [step] Wash your hands and nails with water and soap, then dry them thoroughly on a towel to begin with a clean and dry surface. [substeps] Make sure to completely remove any nail polish you already have on using nail polish remover and a cotton pad, cotton ball, or other preferred method.\",\"endings\":\"[\\\"[title] Press the colored polish onto your nail with a brush. [step] Using a paper towel, gently press the polish into your nails once it's completely coated.\\\", '[title] Coat each nail with the orange-colored nail polish remover. [step] You may wish to let one nail dry for no longer than 5 minutes before applying the orange colored nail polish remover.', 'Working in a dark and ventilated room helps prevent steam from entering the skin while you apply the polish. Turn off the thermostat at least 20 minutes before you apply the polish for the best results.', 'Magnetic nail polish can be applied to the toenails as well. Prep toenails in the same way as fingernails.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}