{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"abf8fb10-0e21-5d0c-8f91-bd602a5c45ec","task_key":"default--validation--44894","task_revision_id":"1","upstream_id":"44894","short_description":"HellaSwag validation 44894","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to set a bulova watch [title] Pull out the crown. [step] Locate the crown at the side of the watch. Pinch it between your thumb and index finger, then lift it straight out to the side.\",\"endings\":\"['The crown should now pop out, hanging from the side of the watch behind the shoulder collar. [substeps] Keep your hand as still as possible while you hold the crown in place.', \\\"Try to remove some of the stiffness. If it's frustrating, the crown should be moved into a new position.\\\", \\\"[substeps] If you don't have a crown, you can also use a nickel-sized device, then slide it onto the shiny side of your watch. [title] Pair the crown with a silver watch.\\\", '[substeps] If the watch has a second hand, wait until the second hand reaches the 60th second mark (the 12th hour mark) at the top of the clock before pulling out the crown. The crown should click into an \\\" out \\\" position without resistance.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}