{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"961d5081-e80b-58fb-99ac-5ced8b368a16","task_key":"default--validation--13192","task_revision_id":"1","upstream_id":"13192","short_description":"HellaSwag validation 13192","config":"default","split":"validation","body":"{\"activity_label\":\"Youth\",\"ctx\":\"[header] How to be a goth (10 and older ) [title] Start with the music. [step] Some good starter goth bands are the cure, bauhaus and the sisters of mercy. But listen to whatever music you like, and don't listen to a band just because they're gothic.\",\"endings\":\"['[substeps] To know some new social styles, try looking into goth bands such as the ones in b.s. if you study goth, see if your school offers that.', \\\"At least wear things like air filters or metal when you're playing them. [substeps] Do goth style events with trend influences or fashion.\\\", \\\"[title] Spend time with the physical sensations while you're driving and dancing. [step] Come up with new ways to make it cool.\\\", \\\"[title] Many goths love books, so go to your library and pick out a few. [step] Edgar allan poe's poetry and short stories are a good start.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}