{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3cd11f11-ccd0-5a9b-b2f7-49a4f76a8d09","task_key":"default--validation--48898","task_revision_id":"1","upstream_id":"48898","short_description":"HellaSwag validation 48898","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to remove your mustache (for girls ) [title] Choose a depilatory cream for a painless option. [step] Because the chemicals involved in this process are harsh, select a cream designed for delicate facial skin. Visit a beauty supply store, and search for a product that specifically states it's for facial hair.\",\"endings\":\"[\\\"If you're unsure, ask an associate to point one out for you. [title] Find a depilatory cream that is appropriate for facial hair.\\\", \\\"[substeps] Depilatory creams expire after 1 or 2 days. It's easy to forget it exists, but note that they'll still remove your mustache once you clean it and reduce its appearance once you've been using it for awhile.\\\", 'It may also say something like \\\" depilatory cream for facial hair. \\\" [substeps] If you have a hard time removing your mustache without using a facial gel, use a lozenge spray.', 'For more severe cases, choose a plastic or glass depilatory cream that has at least spf 30 or more. Some depilatory cream may take longer, though, due to the fact that it will stick more firmly into the skin.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}