{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"20e843ac-f73d-55d7-b06e-bd429cb44415","task_key":"default--validation--27726","task_revision_id":"1","upstream_id":"27726","short_description":"HellaSwag validation 27726","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to keep a diary or journal (for teenagers ) [title] Buy a diary or journal. [step] This is only necessary if you don't have one already. If you want to keep a journal, buy a nice looking journal or you can even use a spiral notebook.\",\"endings\":\"['It is important to have at least 1200 characters to write in. Consider getting a journal that is intended for teens.', \\\"Something that's easy to hold and a few pens and pencils is nice. [substeps] Choose a solid color and don't colour your notebook any sparkly or metallic in the color scheme.\\\", '[title] Mark things that cause a lot of stress to you. [step] Over this amount of time, things like fear of meeting an older relative, worry about things that affect your health, or anxiety about the future can affect your diary or journal.', 'Try bookstores and regular grocery stores in the section with books or school supplies for the spiral notebooks. Consider how you are going to decorate the book when you are buying it.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}