{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"05739ffb-e775-58b2-bf07-e538a40549a5","task_key":"default--validation--46211","task_revision_id":"1","upstream_id":"46211","short_description":"HellaSwag validation 46211","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make a holiday bracelet as a gift [title] Choose a theme. [step] We already have a general theme \\\" holidays! \\\". Although, what type of holidays? Is it maybe easter holidays, or christmas holidays? It is up to you! In this article we will focus on making a christmas theme bracelet, but you can easily change the colours to suit your holiday! [title] Choose colours.\",\"endings\":\"['[step] When deciding on the theme and colour scheme you want, consider. Above all there are options such as bright or cold reds or yellows, gold and purples, and, of course, modern themes.', '[step] Almost any colour you can think of has to be used for you theme. So try to have two different holidays: christmas and easter.', '[step] Colours are very important in a bracelet, and they should suit the theme. Probably the first things that come in mind for \\\" colours on christmas \\\" would be green and red, and maybe gold.', '[step] Here are some ideas : [substeps] Violets are especially festive, and can be anything like flowers, [these are often used in christmas decor]. Red can be anything like corals, accents, etc.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}