{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"22787ffd-6905-5250-969d-ceec8c4ba86d","task_key":"default--validation--28405","task_revision_id":"1","upstream_id":"28405","short_description":"HellaSwag validation 28405","config":"default","split":"validation","body":"{\"activity_label\":\"Holidays and Traditions\",\"ctx\":\"[header] How to make new year's resolutions [title] Be clear about what you want. [step] For example, if you want to take better care of yourself this year, what does that look like? Perhaps you are ready to quit smoking, to take on a consistent exercise regimen, or to eat healthier. [title] Understand why you want it.\",\"endings\":\"[\\\"[step] Why is it important to you to take better care of yourself? Perhaps you want to set a good example for your children or grandchildren. Maybe you're concerned about aging or hereditary diseases, and you're taking preventative measures.\\\", \\\"[step] You're entitled to some particular things, and your resolution is to earn them. By this point, you may not have time for more important goals.\\\", '[step] You have spent a lot of time visualizing changes your life would make, and so you will want to focus on achieving those goals. The important thing is to be clear why you want something more than what you are asking of yourself.', '[step] Before committing to change, be clear with yourself why you want to change your life. [substeps] What does this idea or concept mean? Is your goal a drastic change to your attitude or lifestyle? If yes, then not yet.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}