{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"05b023fa-3840-5dd2-824f-374a6a75b429","task_key":"default--validation--28926","task_revision_id":"1","upstream_id":"28926","short_description":"HellaSwag validation 28926","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to be a virtuous woman [title] Understand that true beauty lies within. [step] True beauty is virtuous because true beauty develops out of good character and caring about the world. Before you consider looking virtuous, you need to look internally so that you actually become virtuous.\",\"endings\":\"[\\\"Every human being is virtuous in his or her own way; up or down the line, on all levels. [substeps] True beauty isn't just in appearance or physical form.\\\", \\\"This means that if you are born really modest or didn't spend much money on such an internal appearance then you could develop a virtuous attitude. [substeps] Being virtuous means that you can be virtuous by turning all the negative feelings into positive ones, which leads to greater moral righteousness.\\\", '[substeps] True beauty lies within what the fabric of your life actually is. An indication of your true nature will differ depending on your inner and outer self-image.', '[substeps] True inner beauty and virtue go hand-in-hand. If you practice the life of a virtuous woman, you will also be a beautiful woman.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}