{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8d174769-ce69-552e-bf16-b776f2260c2e","task_key":"default--validation--45955","task_revision_id":"1","upstream_id":"45955","short_description":"HellaSwag validation 45955","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to choose fall wedding decorations [title] Try soft greens and browns. [step] While greens are usually associated with spring, they can also do just fine for a fall wedding. Think moss or sage green, paired with a soft chestnut brown.\",\"endings\":\"['[substeps] Moss and sage are both brilliant choices for spring weddings. For this reason, they can also help warm up winter weddings, especially since any temperature change or frost slows down the colors of the light fixtures.', 'It can be low maintenance, elegant, soft and interesting. [title] Choose a second color of wedding flower.', 'Avoid bright colors like red, lemon blue, and pink as these tend to cling. Go for falsies or greys instead.', 'These colors give the wedding a woodsy feel without screaming \\\" fall. \\\" [substeps] For instance, you could use natural wood vases with succulents.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}