{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f402b8a9-e247-55cf-a729-f17b4f091658","task_key":"default--validation--35958","task_revision_id":"1","upstream_id":"35958","short_description":"HellaSwag validation 35958","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to slowly but efficiently become a better person [title] If you wish to ease your way into becoming a better person, you must begin by taking a look at yourself. [step] You must learn to notice your flaws and become self-aware of how your actions and words affect others. [title] Begin by asking yourself how you think.\",\"endings\":\"[\\\"[step] Is this your reaction or is it someone else's reaction? Is your reaction such that it is subconscious? These are the reasons why i haven't tried this. If you are over 80 or over, then you might be over 70.\\\", '[step] When did i start thinking like that? Do i actually do it that way? Recognize why you started thinking that way. Not every individual made poor choices, but most people made bad choices.', '[step] This is a number that will help you navigate in the future. There are many ways to help you change; by researching it, you will learn which aspects of your character (in how you (or other person) behave are better (i.e.', '[step] Do you think it is okay to hastily judge someone because of their appearance, ethnicity, sexuality, etc. ? Do you think it is okay to be rude to someone because they are not like you or think differently from you? [title] Ask yourself how, why, and what you speak and say.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}