{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"51488e68-d00a-5ccf-80f6-146a98039c2d","task_key":"default--validation--40008","task_revision_id":"1","upstream_id":"40008","short_description":"HellaSwag validation 40008","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to buy a dishwasher [title] Decide your price range. [step] Before you start searching for a dishwasher, decide a price limit. The lower your price range, the simpler your dishwasher will be.\",\"endings\":\"[\\\"Generally, this is your standard dishwasher that will cost you around $5 per gallon (3.8 us gal) or more. If you've had luck and multiplied by 6 cents, you should be able to find the cheapest dishwasher.\\\", \\\"If you're a filling machine worker, you'll have access to labor-intensive parts. If you're a plumber, though, you may be able to fit more specialized systems into your dishwasher.\\\", 'A good starter may be a sixteen or sixteen-season or twelve-hour dishwasher, which means two cleaners, twice weekly plates, and a dishwasher. A more modern, yet still reliable, dishwasher, a dishwasher with greater demand, and a dishwasher with longer tailings and more storage space will come with a lower price.', 'Budget dishwashers can clean dishes very well, but might not have as many features and might be louder. More expensive dishwashers tend to be quieter and have more features.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}