{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7aa5b666-f4c2-55de-850f-670bbd8f2712","task_key":"default--validation--4893","task_revision_id":"1","upstream_id":"4893","short_description":"HellaSwag validation 4893","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to beat a stronger person in a fight [title] Attack their throat with strikes and chokes. [step] The throat is a weak point in a person's body. When defending yourself, attack their throat by punching or hitting it.\",\"endings\":\"[\\\"Keep this in mind; retaliating can put you on the defensive and could lead to serious serious injury. [substeps] The person's throat might be too narrow to make impacts for anything stronger than you.\\\", 'You can do this either with your right hand or with your left. [substeps] This makes your attack stronger and faster.', \\\"If you're able to get your arm completely around their neck, you can apply pressure to their carotid artery to induce unconsciousness. This artery can be found on both sides of the neck and only requires 11 pounds of pressure to close.\\\", 'Always strike the throat with a strong punch and vice versa. [substeps] A weak point can be as strong or weak as the strike, but is weaker overall.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}