{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"14874715-fade-50f9-9900-181af99fb0ca","task_key":"default--validation--36751","task_revision_id":"1","upstream_id":"36751","short_description":"HellaSwag validation 36751","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to choose a neck tattoo design [title] Decide on a style of tattoo. [step] You may already have a good idea of the tattoo you want to get. However, your idea may require a specific style of tattooing.\",\"endings\":\"['In these cases, a tattoo artist will recommend different designs, such as animal, human, person, or geometric design. [substeps] You can opt for a striped tattoo or a clover tattoo to symbolize a lone person.', 'For instance, if you just want the outline of a star or another simple \\\" flash \\\" design, any experienced, well-respected tattoo artist will likely be able to provide you what you want. However, if you want a certain type of pattern or a realistic image, you may need to go a specific tattoo shop, or even a specific artist, to get what you\\\\'re looking for.', \\\"Ultimately, you may be trying to change the way your tattoo looks, especially if you're trying to stick with alternative designs or use a more unusual style. [substeps] Popular tattoos depicted as iconic and especially unique include polka dots and stripes.\\\", 'If you have a tattoo that turns out famously cool, this may not be the best option. [substeps] Think back to your vision for a tattoo, and see how this style works for you.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}