{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7643ec49-4c3b-5bf2-9e49-7e745324df96","task_key":"default--validation--14953","task_revision_id":"1","upstream_id":"14953","short_description":"HellaSwag validation 14953","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to handle well meaning friends and family when you decide to be a vegetarian [title] Stand firm in your beliefs. [step] Be sure as to why being vegetarian counts for you. When you decide to become a vegetarian, some of your family and friends may not understand or agree with why you're choosing this lifestyle.\",\"endings\":\"[\\\"If you're in a relationship with someone who doesn't understand why you should choose this lifestyle, don't assume they are doing so so because they don't respect your beliefs. Again, it's up to you.\\\", \\\"If you're not sure yourself then it may be possible to be persuaded out of it. [title] Let your friends and family know how you feel and why you want to transition into a vegetarian lifestyle.\\\", 'Consult the following steps to make sure people are just as vegetarian or vegan as you. [substeps] Stay away from any swear words or profanity.', \\\"Either they don't like you, or they don't support the idea of you being a vegetarian. Make sure that you don't put yourself in these positions.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}