{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d96fdfab-19e4-5f8f-af66-d7be6ad4d2dc","task_key":"default--validation--36353","task_revision_id":"1","upstream_id":"36353","short_description":"HellaSwag validation 36353","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to act shy [title] Wear simple clothing. [step] One of the first things people notice is what you are wearing. Put on simple, monochromatic clothes that cover your body to give the impression that you're shy and don't want to expose anything about yourself.\",\"endings\":\"['[substeps] Choose dark and conservative colors like black or navy. Avoid bright colors or prints that draw attention to you like animal prints, red or yellow.', \\\"[substeps] Choose comfortable clothing so that you aren't wearing tight clothes. If you are running errands, it might be a good idea to bring your schoolbag with you, but you can bring it with you when you walk to work so that you don't get caught off guard.\\\", 'For example, make sure you keep your hair out of your face. If you want to be shyness-like, pick out pants and tops with what you think should be appropriate.', '[substeps] For instance, wear a plain bathing suit with a striped button-down or a pair of skinny jeans with form-fitting tops. This is a great way to show your body language without feeling self-conscious.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}