{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"57f22ad5-0d27-5e4a-889f-7fe2db0d8ce2","task_key":"default--validation--32976","task_revision_id":"1","upstream_id":"32976","short_description":"HellaSwag validation 32976","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to ask someone to be your study buddy [title] Reflect on your goals. [step] Ask yourself what kind of study buddy might be good for you, what kind of person you would like to study with, and where you would like to study. [substeps] Make note of your strengths and weaknesses.\",\"endings\":\"['Know what kind of help you can contribute and what kind of help you want to receive. A good study buddy will give help, as well as seek it.', \\\"Maybe you have a few good traits that you would like others to have, like an aptitude for math or strong communication skills. Maybe you're a good study buddy who takes notes because he can write down in a notepad what you say.\\\", \\\"Then, look out for the best friend or significant other. This is important, as when other people ask what's your biggest flaw, they take advantage of the weaknesses, especially the friend with the most strengths, weaknesses, or skills.\\\", 'Define your personal strengths as well as your individual weaknesses. [title] Ask people what they think of your study buddy approach.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}