{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"aa3c84b6-daf9-5cd6-adf6-7a97a1b4f203","task_key":"default--validation--22674","task_revision_id":"1","upstream_id":"22674","short_description":"HellaSwag validation 22674","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to tackle difficult reading material [title] Read the title and think about what it means and which resources might help you (for example italian dictionary or atlas). [title] Underline words you don't know. [step] It is essential to be honest with yourself (after all no one is looking! ).\",\"endings\":\"['Underlining is a practice which can be a terrible sign, rather than a beautiful word. Now write down all of your words that you find the most insightful and bookish.', \\\"It may not be very hard to do so. However you don't have to be such a nerd to look up words you didn't know.\\\", '[title] Write down what you think you know. [step] It can help if you repeat it out loud/in your head multiple times.', \\\"If you cannot use a word in a sentence you don't understand what it means. Imagine yourself in a real-world situation, using the word to describe something to a stranger.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}