{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a1502146-803e-5fea-948e-448409de741b","task_key":"default--validation--12884","task_revision_id":"1","upstream_id":"12884","short_description":"HellaSwag validation 12884","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to dress up your basics [title] Consider getting your basics tailored. [step] If you have a limited wardrobe, making sure your clothes fit really well can make them seem dressier. It can be cheaper to take well-made clothes you already have to a local tailor than to buy a higher class, better fitting wardrobe.\",\"endings\":\"['[title] Add a jacket to dress up a shirt. [step] One simple way to dress up a shirt is to pair it with a suit jacket.', 'Some clothes, like a suit, have small, sensible, statement tags that look nice on you. That means they fit you exactly, look great and are in style.', \\\"If you've already got your wardrobe custom-tailored, you may actually prefer taking quality garments on the trip. These will usually be your staple pieces so don't feel that you have to choose a variety of styles just for them.\\\", '[substeps] Think carefully about the type of dress you want to wear to ensure your wardrobe exactly meets your needs. Mix and match is the best way to determine what suits your image.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}