{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e768f3b5-b81a-5145-ad9f-b8c90c5a4a8c","task_key":"default--validation--27456","task_revision_id":"1","upstream_id":"27456","short_description":"HellaSwag validation 27456","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to defend a punch [title] Present a profile to your attacker. [step] Turn so one shoulder is pointed at your attacker, instead of facing him straight on. Keep one foot slightly toward him, and the other turned out and slightly behind you.\",\"endings\":\"['[substeps] You should stand as close as possible with the heel of your knee directly under his body. Start with you dominant foot toward his back, and your non-dominant foot facing forward.', \\\"[title] Look them in the eye, but don't deliver the punch. [step] Avoid looking directly at them-this may cause them to think you're running toward them, or they may gauge your stance through body language as well.\\\", 'Stay light on your feet so you can move quickly in any direction without losing your balance. Your attacker now has a smaller target, with fewer vulnerable areas.', '[substeps] Try to be friendly, but not pushy. Say \\\" hi, i\\\\'m claire, what\\\\'s your name? \\\" [title] Position the body to make a fist.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}