{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e1ed78c6-ca80-5c03-879d-11520b68c073","task_key":"default--validation--37729","task_revision_id":"1","upstream_id":"37729","short_description":"HellaSwag validation 37729","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to use dead sea salt for skincare [title] Purchase your dead sea salt. [step] There are several online retailers that sell dead sea salt. Companies like seasalt.com, sfsalt.com, and cleopatraschoice.com all sell various sizes of dead sea salts, and you can also search retailers through amazon.\",\"endings\":\"[\\\"If you'd rather purchase in person, places like walmart or target should stock it. Dead sea salt has multiple uses, and pure salt doesn't expire.\\\", '[substeps] Most stores require customers to fill out a credit card at checkout. They offer a range of options, depending on the brand of the dead sea salt you buy.', 'You will use the size under the categories \\\" salt per 1 gallon \\\" and \\\" sea salt per km/hour \\\" here. The average price of dead sea salt includes $10 per gallon (140-260 ml).', \\\"[substeps] Stores like american heart balm, claire's and compostqabcd sell washable sea salt at a variety of prices including hundreds of dollars, depending on how much you use. [title] Spray dead sea salt on your hair.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}