{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3a843ec0-ee2c-5d6e-99d1-088835caf55e","task_key":"default--validation--17037","task_revision_id":"1","upstream_id":"17037","short_description":"HellaSwag validation 17037","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to be a fashion designer [title] Take relevant high school classes. [step] If you're considering a career in fashion design, it's never to early to start exploring your interests and gaining experience. You can even begin your fashion design education in high school, taking relevant coursework that can help you get a head start on your college education.\",\"endings\":\"['You can get technical and technical class opportunities anytime you want. [substeps] Join a crafting group to help you get to know more.', '[substeps] Study as many fashion design principles as you can. Become an international designer if you can in your education.', \\\"[substeps] It's important to get-up start as soon as possible as getting a degree in fashion is one of the more important aspects of that industry. The fact of the matter is, anyone who succeeds all the time has a reason to help a fashion designer gain popularity.\\\", '[substeps] Because designs often start as drawings/sketches, it would be useful to take any art classes your school offers. Many high schools now offer computer-aided design (cad) classes.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}