{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"dfb08f53-6f31-546c-b9af-c88e417f9844","task_key":"default--validation--27234","task_revision_id":"1","upstream_id":"27234","short_description":"HellaSwag validation 27234","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to wear a denim vest [title] Choose a color. [step] Choose a color of denim that's right for you. Vests come in many shades of blue, a variety of blacks and even whites.\",\"endings\":\"[\\\"You can mix both colors, or pick a color that's opposite or complementary. The first color is for example light blue or brown, while the second color is orange.\\\", 'Denim is typically available in several types of washes as well to really add a variety of color. Choose one that best suits your tastes and personality.', 'The darker blue is best for winter, and the darker grey is best for summer. [substeps] If you do not want to invest in a vest, you can find one specifically made for the fall by nike.', 'Regardless of what kind of color choice you purchase, steer clear of ones in the color range of that style. [substeps] For instance, a teal, cream, or black plaid vest would look great against a light yellow statement on a white tank top or black juicy skin short shorts.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}