{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cfb0deee-bbed-5104-b5bd-47fb6e5e7ae4","task_key":"default--validation--41573","task_revision_id":"1","upstream_id":"41573","short_description":"HellaSwag validation 41573","config":"default","split":"validation","body":"{\"activity_label\":\"Cars & Other Vehicles\",\"ctx\":\"[header] How to register a car in florida [title] Check your florida license. [step] You can't register a car in florida unless you have a valid florida id, so make sure your card is updated and unexpired. If you've just moved to florida from another state, have your old license.\",\"endings\":\"['If you purchased a replacement specific to your business, bring it back with you when you leave. This will provide you with your updated identification.', 'Look in your papers for the replacement certificate. You should have gotten one when processing your paperwork.', '[substeps] You can register a car in your name if you only have a permit, but you have to be at least 18 years old. [title] Make sure you have florida insurance for your car.', \\\"If you don't have a renewal card, you can apply online. [title] Contact your driving school to get a new driving permit.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}