{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4940f46e-c761-551d-8515-cac4a4d08154","task_key":"default--validation--42830","task_revision_id":"1","upstream_id":"42830","short_description":"HellaSwag validation 42830","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to wear a pashmina [title] Wrap your pashmina loosely around your neck. [step] Unfold your scarf so that all of the material is exposed. Then, wrap the scarf around your neck once.\",\"endings\":\"[\\\"[title] Unravel your pashmina. [step] Next, you'll need to unravel the ends of the scarf and use your hands to take apart your subconscious pashmina.\\\", '[title] Place your right hand under your left chest and your left hand over your stomach. [step] If your right hand is still clasped after wrapping your pashmina, it should be cupped over your left shoulder, but not pushed up.', 'The longer you wrap the scarf around your neck, the more cohesive it will be. Make sure that the scarf is snug enough that none of the remaining material will tear or rip.', 'Tighten the loop around your neck, or leave it loose and hanging near your bust. You can choose to have the tails hanging in either the front or around your back.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}