{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e6b7264d-6f1b-507c-9702-3bd119a2cdfb","task_key":"default--validation--12053","task_revision_id":"1","upstream_id":"12053","short_description":"HellaSwag validation 12053","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to feather your hair [title] Learn what feathering is. [step] A feathered hairstyle is a look that is created with layered hair, with smooth layers that flip out and back looking almost like the overlapping feathers on a bird. The hair is usually parted down the middle and then curled back away from the face.\",\"endings\":\"['[substeps] Feathered hair includes two feathers if you have spiky or layered hair, two feathers if you are long, and two feathers if you are short. Try to use a feather that fades fairly easily after you use it.', 'However, it can also be teased or knotted around one side of the head or clipped at the front. [title] Wear feather skirts, headbands, hats, and gloves.', 'It requires a finely-textured layered hairstyle and can be created from short and smooth hair or long and curly. However, if you feather curly hair, the feathers will only be visible when you straighten your hair.', 'The feathered parts of the hair are pinned back with clip in at the temples, and are often worn with aviator sunglasses. In actuality, feathered hair is made up of softer hairs located on the top layer of hair about half an inch larger than the previous layer.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}