{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9f2af83d-4743-5800-9c42-867b4d896f68","task_key":"default--validation--30133","task_revision_id":"1","upstream_id":"30133","short_description":"HellaSwag validation 30133","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to use a hair mask and sea salt to make beach style waves [title] Add two egg yolks, honey, melted butter and milk in a jug, mixing bowl or other suitable container. [title] Mix together to combine. [title] Make the sea salt spray.\",\"endings\":\"['[title] Spray the mixture from head to feet. [title] Using a curling wand or a curling iron form the sea salt spray into waves.', '[step] Use only sea salt, not table salt. Add one to two teaspoons of salt to a spray bottle filled with water.', '[step] In small bowl mix the salt and spread evenly throughout your hair using a brush. [substeps] Washing water from hair washes and shampoos is optional.', '[step] Place in a bowl or other container. Spray the mixture all over your hair, back and neck (including your face).']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}