{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e650d869-c4c5-56d8-8899-21c9a6f59012","task_key":"default--validation--2737","task_revision_id":"1","upstream_id":"2737","short_description":"HellaSwag validation 2737","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to wear bright colors in winter [title] Go neon. [step] If you love bright colors, consider a coat or jacket in a neon color, such as bright pink or electric blue. Make sure the rest of your outfit consists of mostly dark neutral pieces, which will balance out the loudness of the neon and keep the overall effect winter-appropriate.\",\"endings\":\"['[substeps] For instance, you could pair a neon pink coat with black skinny jeans and a simple black and white striped top. Wear a neon green pea coat with basic blue jeans, a black sweater and statement heels.', 'Go for neon colors or neutral, solid colors in winter, though. If you start to feel nervous, take a minute to sit and consider the color blues, pinks, greens, or blues.', '[substeps] In general, if you like wearing people patterns, wear a brightly-colored coat or jacket. Make sure to keep your outfit light and airy to keep the sun from shining on your face, but be sure to keep your decorations in place too.', '[substeps] You can wear vibrant, clashing pastel colors, or bold, glossier colors. Make sure that the colors complement one another well, and vice versa.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}