{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8e0702f0-6027-537b-af19-4f79bf04c748","task_key":"default--validation--30760","task_revision_id":"1","upstream_id":"30760","short_description":"HellaSwag validation 30760","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to use pedicure tools [title] Remove nail polish with cotton balls. [step] Before you give a pedicure, make sure to remove any old nail polish from your toenails. Apply some acetone nail polish remover to a cotton ball and swipe it over your toenails until the nail polish is completely gone.\",\"endings\":\"[\\\"Gently reapply acetone to each toenail and then let the polish dry. [substeps] You may need to reapply an additional layer of nail polish remover and also grab some cotton balls if you're using nail polish remover to remove polish from your toenails.\\\", 'You should wipe off any salt and petroleum that was left behind. [substeps] Get in a warm, safe place to have your pedicure.', '[substeps] Removing your nail polish will give your nails a chance to breathe. It will also allow the water to soften the nails and allow you to smooth them out with a nail buffer.', 'Then, pick up a small section of your toenail to smooth it out. [substeps] You can always cut off more polish and manually smooth it out later.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}