{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f60ea27e-0ad6-570b-9afa-6ffdb7f122cb","task_key":"default--validation--36912","task_revision_id":"1","upstream_id":"36912","short_description":"HellaSwag validation 36912","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to act high [title] Master the zoned-out stare. [step] People who are high often laze around, watching the world around them through half-closed lids. To perfect the dazed and half asleep look, slack your jaw and loll your head slightly to one side.\",\"endings\":\"['[substeps] People who smoke or consume marijuana often get lost in their thoughts, and are easily distracted. To really show you have zoned out, act like you are thinking about something totally irrelevant to your surroundings.', \\\"Close your eyes, but resist the urge to open them in the process. [substeps] Slouching, mirroring other people's bodies language when you're not in your shell will help you blend into a more welcoming environment.\\\", 'This maneuver will help free your jaw, which is a sign of open mouth and open mouth. [substeps] Keeping both eyes closed, put two fingers under your chin, and allow the palm to hover over your mouth.', \\\"On the other hand, you can open your eyes slightly, and blink them a few times, without being worried. For some people this is difficult, but for others it's perfectly normal.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}