{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"64ca2852-3936-5e17-85ff-77439e2b354c","task_key":"default--validation--8615","task_revision_id":"1","upstream_id":"8615","short_description":"HellaSwag validation 8615","config":"default","split":"validation","body":"{\"activity_label\":\"Youth\",\"ctx\":\"[header] How to become a scene queen on facebook [title] Get the hair. [step] The hair is the most important step. Make sure to get lots of layers and if your hair is short get some extensions.\",\"endings\":\"[\\\"You don't have to wear them all the time, just for some of your pictures. Most scene queens on facebook will have varying hair.\\\", \\\"[title] Don't care if this is your first time getting it, if you want it then ignore it. [step] You will start figuring out who you are, and this will not make much sense to begin with.\\\", 'If you have long hair add a couple of layers per 2/3 or side of your head. If you have hair that is shoulder length try to get extensions usually put more layers.', 'When you go to school, try to only have your hair in layers. If you have your hair in braids, just skip this step.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}