{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c95fc9ad-18df-5cf8-9521-4b076b170ade","task_key":"default--validation--49612","task_revision_id":"1","upstream_id":"49612","short_description":"HellaSwag validation 49612","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to dress for a date (for teen girls ) [title] Check the weather. [step] Is it going to be warm and sunny or cold and wet? Look at the hourly weather report for the time you expect to be gone on your date. This will help you know what kind of clothes to wear as well if you may need additional items, such as an umbrella or sunscreen.\",\"endings\":\"['[substeps] For warm weather, consider a sundress with cute sandals. You could also opt for a pair of shorts and a nice top; a shirt with a little sparkle or shine would dress up your shorts well.', \\\"[substeps] You should wear contacts to prevent squinting and make it easier for you to see you're talking. [title] Wear a shade jacket or a hat.\\\", '[substeps] If there are a lot of different types of clothing, you may need to make your wardrobe and accessories your top priorities. Alternatively, you may want to look for outdoorsy clothing ideas that will work for both of you.', 'Bring your best clothes and plan to be self-sufficient. [substeps] If you plan to go out on the town, bring your cell phone, credit card, wallet, and any other valuable items.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}