{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"281863dd-7b92-5e2e-8da9-eb421dad30ae","task_key":"default--validation--22764--variant-bf62a7098842","task_revision_id":"1","upstream_id":"22764--variant-bf62a7098842","short_description":"HellaSwag validation 22764","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to drive a van [title] Adjust your seat and mirrors. [step] Move your seat until you can reach the pedals comfortably while still seeing all your mirrors. Adjust your mirrors so that you can clearly view the adjacent roads and just a small amount of the van's edge.\",\"endings\":\"[\\\"If you have motion sickness, you may want to sit a bit farther back than you normally would while driving van. You'll want to do some preliminary maintenance to make sure that your vehicle's airspace is as quiet as possible.\\\", \\\"Since they're designed to hold cargo, many vans do not include a rear-view mirror, making your side mirrors more important. [substeps] Some vans have extendable side mirrors designed for people carrying trailers.\\\", \\\"[substeps] Make sure that the front fender area of your vehicle comes completely above the seat's height. Also, avoid tilting the front fender forward too much or the car will make a wide angled curve in your direction.\\\", '[substeps] If possible, make sure that you are facing forward in traffic. Try to angle it as an\\\\' inverted\\\\' \\\" v \\\" (not a \\\" s \\\") to see which side of your seat will be more visible from your side mirror.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}