{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"11a7e98c-4412-5d84-831e-7f0c5e346d8b","task_key":"default--validation--41551","task_revision_id":"1","upstream_id":"41551","short_description":"HellaSwag validation 41551","config":"default","split":"validation","body":"{\"activity_label\":\"Cars & Other Vehicles\",\"ctx\":\"[header] How to find your vin (vehicle identification number ) [title] Check the dashboard. [step] Most vin plates appear on the lower left-hand side of the dashboard. Sit in the driver's seat and look on the dashboard in front of the steering wheel.\",\"endings\":\"[\\\"You should find the vin on the left side of the dashboard. [substeps] If you're not having trouble finding the vin, do not use the drop down menu.\\\", 'If the wheels themselves seem to move about, you may need to dip down into your seat to see just underneath the wheel. Pay attention to the signals appearing on the dashboard.', 'Alternately, you might be able to see the vin easier by standing outside the vehicle and looking through the windshield. [substeps] The vin should have 17 characters if the car was manufactured after 1981.', \\\"[substeps] If you know which one you're driving, double check your vin regularly. If you have a tendency to have friends bleed out gas on the road, pay attention to this.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}