{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5448e5ec-20db-5cd9-b6c5-2fd18d734490","task_key":"default--validation--48045--variant-1623b50a78a1","task_revision_id":"1","upstream_id":"48045--variant-1623b50a78a1","short_description":"HellaSwag validation 48045","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to avoid male stereotypes [title] Complete a bias test. [step] Most people have some sort of bias. That means that you likely believe in some stereotypes, whether you know if or not.\",\"endings\":\"['You can take an online test to see what your personal biases are. Harvard university offers a variety of free, online tests.', 'However, many of them are based on coercion or misinformation. That means that you are unlikely to have your bias confirmed on the first try.', \\\"In this case, you're better off thinking of stereotypes as facts rather than mere myths. Stereotypes are reality, not weeds and weeds.\\\", \\\"You may imagine that you're right-handed or left-handed, or both. All of those stereotypes need to not apply to you.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}