{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7d0724e4-4d46-5e53-a45a-88bc1b560aea","task_key":"default--validation--28258","task_revision_id":"1","upstream_id":"28258","short_description":"HellaSwag validation 28258","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to drive a stick shift with a broken left foot [title] Get in the vehicle. [substeps] Lean your crutch up against the side of the car, right behind the driver's side door. Unlock the door while maintaining your balance, and carefully slide into the driver's seat.\",\"endings\":\"['Start wrapping your left foot around the front wheel to bring the crutch up and in. This will give you the best possible grip.', 'Your feet should be in contact with your hazard board, and your knees should be bent at 90 degrees as you lean into the car. Using your left leg, lean and center your weight onto your right leg.', \\\"To avoid further injury, it is advised to slide in with your backside first. If you are using a second crutch, place it on the passenger seat or behind the driver's seat.\\\", '[title] Mount your snare and turn it on. [substeps] Grip both of the rear tires while rotating the snare counterclockwise.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}