{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1ad8d1a2-7eec-5983-833b-9061c2e5bd82","task_key":"default--validation--3618","task_revision_id":"1","upstream_id":"3618","short_description":"HellaSwag validation 3618","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to pierce your ear [title] Use pre-packaged 70% isopropyl alcohol swabs to clean your ear. [step] You must do this so that your ear is completely wiped clean of any bacteria that may have gotten into your piercing. Wait until the ear is dry to pierce it.\",\"endings\":\"['Because there will be a thin layer of bacteria and dead skin on your ear, the finished site of your piercing will be roughly 100% bacteria. [title] Remove the outer layer of the earwax.', '[substeps] The volume of alcohol you use should be around twice as much as how long it will take the needle to sink into your eardrum. Otherwise, the needle could break the ring.', '[substeps] You can also use hydrogen peroxide or rubbing alcohol to sterilize your ear. [title] Make a mark where you want your piercing (s) to go.', 'Isopropyl alcohol will dissolve any bacteria and will allow the swab to fully dry. Avoid standing directly over your piercing, as this may cause additional bacteria to grow in your ear.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}