{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bea5c7d6-63e9-59c5-8465-1c5b347f8565","task_key":"default--validation--32143","task_revision_id":"1","upstream_id":"32143","short_description":"HellaSwag validation 32143","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to buff your nails [title] Remove old nail polish. [step] Clean, polish-free nails are essential before you begin filing. Dip a cotton swab in nail polish remover.\",\"endings\":\"[\\\"Polish remover sticks well to nails, while cotton swabs absorb oil and flaking surface oils. Dip a cotton swab in nail polish remover then dab at the polish's surface, wetting the cotton swab in the process.\\\", 'Stuff your hand, head or foot in the swab and wait for it to work. [substeps] Although nail polish remover is available at many stores, you may want to try finding nail polish remover specifically made for buffing nails.', 'Rub each nail until all of the polish is removed. You may need to go over each nail again with a clean cotton swab to remove traces of polish around your nail and on your skin.', 'Rub it back and forth between your dry hands for several minutes. Any old polish left behind will flake off during the process.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}