{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"20a6ce38-4dfb-5537-9f4a-4502799be19d","task_key":"default--validation--21174","task_revision_id":"1","upstream_id":"21174","short_description":"HellaSwag validation 21174","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to feel amazing [title] Take care of personal hygiene. [step] Taking care of your body and having a little \\\" me \\\" time will help you to both look and feel amazing. [substeps] Take a nice, long, hot shower.\",\"endings\":\"['You will want to shower daily, and use deodorant and perfume daily. Smell your favorite perfumes in the morning and evening.', \\\"Let your hair air dry and take a bath if you don't have time to do so on the weekend. Leave dirty clothes in the hamper or throw away dirty clothes.\\\", 'Make sure to thoroughly wash your hair with shampoo, and leave your conditioner in for 5 minutes. Wash, rinse, and repeat as many times as you desire.', 'Do not let it sink in that this is uncomfortable, it is light and refreshing, gentle. Use some deodorant and a nice smelling perfume.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}