{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"05d84e68-b41e-59e5-8f63-3f3b82c2a0a4","task_key":"default--validation--12820","task_revision_id":"1","upstream_id":"12820","short_description":"HellaSwag validation 12820","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to do a makeup parody [title] Start with your foundation. [step] The key to a makeup parody is to be hilarious. So, when you apply your foundation, use something that is five times darker than your normal skin color.\",\"endings\":\"['If it is a pale pale shade, use bb cream. If it is beige or tan, use latex.', \\\"Don't use mascara, which may make your lashes look stringy. Blush the rest of the way, then apply blush and black eye shadow.\\\", 'Then, fill your cheekbones in with a pink stain. Blow out the tip of your nose only.', 'Say something like, \\\" first, you want to start with a nice, tan foundation that gives you a really glowing complexion. \\\" have a hint of sarcasm as you say this.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}