{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"eafe941b-ea76-5dda-83bb-3912c3ad5da7","task_key":"default--validation--13427","task_revision_id":"1","upstream_id":"13427","short_description":"HellaSwag validation 13427","config":"default","split":"validation","body":"{\"activity_label\":\"Cars & Other Vehicles\",\"ctx\":\"[header] How to use the rear view mirror [title] Position your seat. [step] Before you adjust your mirror, you want to make sure that your seat is in the right position for driving. Move the seat forward or back, adjust the seat back, and move the seat up or down, if your seat has that type of adjustment.\",\"endings\":\"[\\\"[substeps] You may need to experiment, but usually, most seats will adjust the mirror for you. For example, if you can adjust the wheel so that it faces slightly forward, that's a great move for a car with a full rear view mirror.\\\", '[substeps] Most cars have the mirrors checked at their appropriate position prior to deployment. [title] Find your pre600 angle angle.', '[substeps] The outside parts of the body of the car, such as the pedals and headlights, should be aligned. Further, you should check your speedometer or your mirrors to see if you are in the right position.', \\\"[substeps] It's important to adjust the seat so that you can easily reach the pedals of the car. Make sure your feet can reach the gas pedal, the brake pedal, and, if you have one, the clutch pedal.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}