{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"81a26a6e-90c5-54be-b31e-ab89202483bc","task_key":"default--validation--20592","task_revision_id":"1","upstream_id":"20592","short_description":"HellaSwag validation 20592","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to create family unity [title] Use effective communication skills. [step] Each member of the family should feel important and valued. This includes you.\",\"endings\":\"[\\\"Make sure you're in compliance with the requirements for your relationship. [substeps] Let each member have room to speak and be sensitive.\\\", 'Try not to engage in teasing, cursing, or emotional outbursts. You need to treat others as you would treat your own.', 'In order for your family members to hear your needs, you have to express them. Be polite, but honest with your family during discussions.', 'It is important that each member of the family stay informed regarding what is going on and who is still alive. The ability to socialize, communicate, and be an integral part of the family makes you significant.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}