{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b45dd5eb-538b-56f1-9069-80c3221b78d9","task_key":"default--validation--15758","task_revision_id":"1","upstream_id":"15758","short_description":"HellaSwag validation 15758","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make fake dimples [title] Wash the bottle cap. [step] You can use either a metal or a plastic bottle cap, but since the cap will be going into your mouth, you need to make sure that it has been thoroughly cleaned, first. [substeps] Metal bottle caps are shallow and tend to produce more subtle, natural looking fake dimples.\",\"endings\":\"['Plastic caps are generally thin and relatively wrinkle-free. Plastic caps work best on hard-to-reach places, like the grooves on metal bottles.', 'Plastic bottles that have been folded in half or moved around will produce a much more subtle, flattering impression. Plastic bottles that come in two most common colors will also work well.', 'Plastic bottles that are professionally cleaned are far less likely to produce noticeable fake dimples. [title] Moisten a cotton ball with water and wash the bottle.', 'The ridges can be painful, though, especially if you have sensitive cheeks. Plastic bottle caps come in a few different sizes.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}