{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b90b80dd-1998-5ba9-a19c-bbb9d7903407","task_key":"default--validation--28987","task_revision_id":"1","upstream_id":"28987","short_description":"HellaSwag validation 28987","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to style medium length hair [title] Create loose waves. [step] On days where you want to look like you just rolled out of bed, try adding natural waves to your hair. You can do this by twisting big sections of your hair around a large curling iron.\",\"endings\":\"['Most waves will be a single strand, so let your hair hang loose when you step out of bed. [substeps] If you plan on waves all over your body, you may want to start with a loose wave.', 'Use one large section for a very curly wave, and just massage the ends with your fingers to create about an inch or two of wave. [substeps] Alternatively, you can simply leave your hair down in the usual messy ponytail.', '[substeps] Let the hair slide out of the curling iron almost immediately. This will add waves to your hair, instead of ringlets.', 'Let them hang down, flip your head upside down and secure the ends with bobby pins or hair clips. Over time, your waves will grow from there.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}