{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fb521022-b89c-5c8e-a56b-f1e199aaf6e0","task_key":"default--validation--34584","task_revision_id":"1","upstream_id":"34584","short_description":"HellaSwag validation 34584","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to dye a silk scarf [title] Create pleats in the scarf. [step] Fold a white silk scarf in any way that you like to create random pleats in the fabric. Use two rubber bands to loosely secure the pleats at either end of the scarf.\",\"endings\":\"[\\\"As you do this, you should notice that there are no tears left behind after the pleats have been secured. This is normal! [substeps] As for pleats, create them where you'd like the scarf to be.\\\", '[substeps] Allow the scarf to lay on a flat surface so it wrinkles up instead of has momentum when folded. [title] Stuff the scarf with small tulle.', '[substeps] If you prefer, you can twist or roll the scarf to create random pleats. The pleats help create a textured look when you apply the dye.', \\\"[substeps] Don't worry if the scarf drapes over the perimeter of the fold, as long as it covers the fold. [title] Consider wearing a heavy chain to train the pleats.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}