{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c43aa913-dea1-5b3c-80a9-7aa060cb9d30","task_key":"default--validation--34061","task_revision_id":"1","upstream_id":"34061","short_description":"HellaSwag validation 34061","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to change a watch battery [title] Find the small indentation on the back of the watch. [step] Turn the watch over and find the small hole or indentation that's on the side of the watch, between the watch back and the watch itself. This indentation is created specifically to help you pry off the back of the watch.\",\"endings\":\"['[title] Press down on the plastic cover of the watch with your thumb. [step] Only doing this can loosen the plastic.', \\\"[substeps] Skip this step if you'll be removing the watch battery. You can reattach the watch battery later by using the small indentation.\\\", \\\"[substeps] If you can't find the indentation, scan the backside of your watch with a magnifying glass. Wear a pair of dust free latex gloves when replacing your battery.\\\", \\\"[substeps] Newer watches have small, soft indentation that's easy to penetrate. You might also need to connect your fingernail to one of the small holes on the back of the watch.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}