{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"85ebfafd-6fb8-537e-a4a8-5e599cf60016","task_key":"default--validation--34355","task_revision_id":"1","upstream_id":"34355","short_description":"HellaSwag validation 34355","config":"default","split":"validation","body":"{\"activity_label\":\"Travel\",\"ctx\":\"[header] How to tell an interesting travel story [title] Keep an updated log of your travel adventures. [step] Make it a habit to keep a notepad so that you can jot down different experiences you have while you travel. This will make it easier to remember experiences that you can later use in your stories.\",\"endings\":\"['[substeps] Notice which places you are most likely to see in your travels. Read stories that you have never heard of.', \\\"[substeps] Keep it on your phone or tablet, or on your computer if your cellphone is sim-safe. Write down at least one trip or possible trip every week, whenever you're thinking about them.\\\", 'Sign up for a travel journal with descriptions of places you have visited. This will help you remember detailed details without giving away too much information.', 'Jot down the most important points of the story rather than trying to chronicle all the details during your travels. [substeps] You can also take photos or video of your experiences to help you remember them, or you can even integrate them into the telling of your story.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}