{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f98c1f0d-03c3-51e4-be11-0d1704c2e03a","task_key":"default--validation--10870","task_revision_id":"1","upstream_id":"10870","short_description":"HellaSwag validation 10870","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to beat a \\\" tough \\\" person in a fight [title] Make the first move. [step] Don't ever wait for someone to attack first. If you sense an impending attack, hit first and hit hard.\",\"endings\":\"['Use whatever strikes that will be most effective. The key is to get the upper hand, either by surprising them with a punch, by being the first attacker and smashing them to the ground, or by smashing them in the shins with the elbow.', 'This will surprise your attacker and help to level the playing field. [substeps] Consider the legal consequences of your actions.', 'Hitting is the first level in fighting and it essentially gives the opponent a clear direction to running away and just plain wiping out. Stand tall, be confident, and end the challenge in a fast, powerful, and effective way.', \\\"If someone attacks, don't hesitate or hesitate to step back and take their place. [substeps] Don't respond to pop of the head.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}