{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d4305757-906d-5be8-9562-bf2b3df96394","task_key":"default--validation--33298","task_revision_id":"1","upstream_id":"33298","short_description":"HellaSwag validation 33298","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to apply a wig [title] Pick a wig type. [step] There are three major kinds of wigs: the full lace, the partial or front lace, and the non-lace. There are also three main materials they can be made out of, which are human hair, animal hair, and synthetic hair.\",\"endings\":\"['[substeps] The human wig should be very light in color and comparable to human or synthetic hair. Synthetic hair can be dyed like human hair, but it is easier to achieve.', '[substeps] Formatting is the art of twisting, clipping, and pulling wigs together. It is extremely dangerous because you will be breaking the wig apart.', 'Talk to your primary care doctor or a professional you trust. Ask about parts of the wig you like for inspiration or advice.', 'Each wig type has its pros and cons, so make sure you buy the wig that is best for you. [substeps] The full lace wig is made with a lace mesh cap and the hair sewn into it throughout.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}