{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9cbf86c7-61a2-50c9-b8a3-6e844aae358e","task_key":"default--validation--32381","task_revision_id":"1","upstream_id":"32381","short_description":"HellaSwag validation 32381","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to clean a stainless steel watch [title] Remove the watch from the bracelet. [step] Different stainless steel watches require different bracelet detachment methods. Some will need only a simple click of a button to detach their bracelet from their watch head.\",\"endings\":\"[\\\"Other watches require a combination of x's and y's to successfully complete their bracelet removal operation. Check the display tags on your piece of jewelry according to the date on your watch.\\\", \\\"That's it! If yours doesn't have a snap button, it's easy to unscrew or detach a snap wrist device before removal. [substeps] Unscrew the clip from the snap wrist device once it's removed.\\\", 'Others will require extensive wiring and more complicated software. [substeps] Twist off the screw or clip holding the watch head in place.', 'Others will need a specialized screwdriver to release the bracelet from the watch head. Consult manufacturer directions for more information about how to remove the watch from the bracelet.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}