{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5dbcb77a-4c77-51b1-acb8-815c92e4c9c5","task_key":"default--validation--33923","task_revision_id":"1","upstream_id":"33923","short_description":"HellaSwag validation 33923","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to see yourself as you really are [title] Write out your perceptions. [step] Pick up a pen and paper and write out a description of yourself. Make it as detailed as possible, defining yourself as a whole person: physically, mentally, emotionally, and spiritually.\",\"endings\":\"['[substeps] Start with statements like, \\\" i am...\\\" or \\\" what i\\\\'m really proud of myself for is...\\\" answer each statement with at least 8 to 12 responses. List your strengths and weaknesses, as well.', 'Take this in turn to express the real you. [substeps] Write down a few simple steps, such as how you will dress, how you know when to eat (and where), how you feel physically, and so on.', 'Write down all of the major characteristics, personality traits, interests, hobbies, and passions. [substeps] Highlight the most important element for those you think are able to relate to you.', \\\"This will force you to be more imaginative when you write down the characteristics of yourself that you are seeking, and others will see you as you actually are. [substeps] Your description doesn't have to be complex.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}