{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a4498465-efeb-5250-a576-bb51088dce23","task_key":"default--validation--21884","task_revision_id":"1","upstream_id":"21884","short_description":"HellaSwag validation 21884","config":"default","split":"validation","body":"{\"activity_label\":\"Cars & Other Vehicles\",\"ctx\":\"[header] How to change your address on your driver's license [title] Update your address without delay. [step] Your state probably gives you less than a month to report your new address to the department of motor vehicles (dmv). Check online.\",\"endings\":\"['Again, keep in mind that you will have to file a new tax return, not your new home address. [substeps] If you were in a previous location while your new address was updated, you may want to extend the transfer to a different address.', \\\"[substeps] If you don't have your new address listed on your license plate, then call dmv. Alternately, you could replace your address with someone else's.\\\", 'For example, in vermont, you must report your new address within 30 days. In illinois, by contrast, you get only 10 days.', '[title] Ask the department for a new address. [step] One of those and many other places you may have to fill your name change license plates by state change get an id card for your name change.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}