{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5eac1bd1-88d5-5be4-a9bc-9088eb64777e","task_key":"default--validation--10586","task_revision_id":"1","upstream_id":"10586","short_description":"HellaSwag validation 10586","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to drape your lehenga dupatta [title] Make narrow pleats in your lehenga dupatta. [step] Fold your dupatta lengthwise to create narrow pleats. Then, position and pin your dupatta onto your right shoulder.\",\"endings\":\"['[title] Fold the dupatta from front to back, creating a square fold. [step] Use your end result to create a square square on your right shoulder.', 'Keep the pleats on the right side of your shoulder. [title] Pin the dupatta to your navel.', '[title] Cross your right ankle over your left foot. [step] Using your free hand, cross one leg over your left ankle.', 'Lay your pleated lehenga dupatta over your right shoulder so that most of the excess material hangs down in back. The material in front should hang down to your hip.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}