{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"db64096f-e026-5be4-8c51-c500e2c43b84","task_key":"default--validation--33055","task_revision_id":"1","upstream_id":"33055","short_description":"HellaSwag validation 33055","config":"default","split":"validation","body":"{\"activity_label\":\"Cars & Other Vehicles\",\"ctx\":\"[header] How to redeem your repossessed car [title] Find your deadline. [step] After the lender repossesses the car, it should send you some notices in the mail. One notice should inform you of how you can redeem your car.\",\"endings\":\"['[title] Determine how much money you have saved. [step] If you want to redeem your repossessed car, you might want to shop around for repossessed cars so that you can save as much as possible.', \\\"The notice should also tell you the deadline for redeeming the car and the redemption amount. If you don't receive this notice within five days of repossession, then contact the lender.\\\", 'Typically, the notice should mention whether the lender repossesses the car as a new or used one. The notice should also say what the dates are for purchasing a repossessed vehicle in the future.', \\\"These notices must include dates you will redeem and instructions for getting the car's repossessed vehicle. [substeps] See method 1 for detailed instructions on how to redeem a repossessed car in the future.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}