{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"94c45f49-431c-5f42-8579-8a0d46fa5b10","task_key":"default--validation--41681","task_revision_id":"1","upstream_id":"41681","short_description":"HellaSwag validation 41681","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to be cute [title] Be positive. [step] Cute people are always, and spread their joy for life to others. A cute person's warmth should be able to fill a room and should be felt from a mile away.\",\"endings\":\"['If you want to improve your positive nature, there are a few things you can do : [substeps] Smile. Do not look like a goof when you smile; just smile like you always do.', 'Next to adorable, likable, and kind, there are around fifty cute qualities that attract and draw every person to the best of their abilities, from the positive to downright frightening. [substeps] Cheer up people by saying: \\\" i think you are such a cutie and care about how you look! \\\" remind people to stop if they are having a terrible day.', '[title] Give compliments when you have to be cute. [step] Unfortunately, being cute is essentially a flaw-everyone has such flaws (but not many people like them! ).', 'If you focus on being positive, your energy will be directed elsewhere and other people will not be able to do the same. [substeps] From a young age, a cute person must be able to talk to and connect with people from all walks of life (from parents to siblings).']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}