{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a074fbcc-e56b-5270-ae25-b97e816ab86f","task_key":"default--validation--6218","task_revision_id":"1","upstream_id":"6218","short_description":"HellaSwag validation 6218","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to avoid getting beat up by a bully [title] Avoid the bully or bullies. [step] Many bullies rely on setting up their victims for embarrassment, or hang out in a specific area of the school yard. Look out for places where bullies may congregate (e.g., in a secluded area behind a remote campus building) and avoid these locations.\",\"endings\":\"['When a bully is harassing you, do not engage in any sort of physical contact with them-physically or through his words or body language. [substeps] Bullies can take things public, before you can block them.', '[substeps] Avoid stalking or tagging bullies. Bullies rely on intimidation from teachers, supervisors, and other students to approach students and harm them.', '[substeps] Instead, stay away from places that have posted signs and posters that state \\\" stand back \\\" on doors or at least \\\" ignore \\\" on ceiling boards. If there is one in one area, stay at it.', '[substeps] If you are not near the bully, you will not become a victim. [title] Stick with your friends to avoid being trapped by bullies.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}