{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9f1a7631-ce11-59c5-bc98-46aa68a059a5","task_key":"default--validation--47452","task_revision_id":"1","upstream_id":"47452","short_description":"HellaSwag validation 47452","config":"default","split":"validation","body":"{\"activity_label\":\"Youth\",\"ctx\":\"[header] How to run away from home as a teen [title] Have a good reason for leaving. [step] Run away for kicks, adventure, or to teach your parents a lesson. Living on the streets is difficult, and nature will not provide you with blankets when you are cold or food when you are hungry.\",\"endings\":\"[\\\"If you are being abused or neglected, the safest solution is to call children's aid or the police before trying to run away. [substeps] Some teens, especially those with troubled families, think that foster care is worse than running away.\\\", \\\"Have a good reason for leaving, so that your parents can find you hiding. [substeps] Don't be embarrassed about your break-up.\\\", \\\"[substeps] Tell your parents why you are going away. Don't worry about exactly telling your parents the truth.\\\", 'So, being on the streets will show parents that you are motivated to leave the place of your choosing. [substeps] Think about whether you plan to focus on family support, teach others, or even leave your home to run away.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}