{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"51a43c45-cacd-535d-b3b4-f6c1ae32cf80","task_key":"default--validation--17118","task_revision_id":"1","upstream_id":"17118","short_description":"HellaSwag validation 17118","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to use nose hair removal cream [title] Read all labels. [step] When you shop for nose hair removal cream, you must be very careful about what you purchase for this purpose. Most hair removal creams are not intended to be used in the nose because their odors can cause dizziness and nausea.\",\"endings\":\"['Read labels carefully to be certain that the cream you get does not specifically say to avoid using it in the nose. [substeps] The truth is that no hair removal cream is designed for the nose, so you should be very careful if you find a cream that does say it is safe for the nose.', 'In order to avoid catching these side effects, read all labels on the bottle with the name of the product included. An inventory of the product can also be helpful : [substeps] Any types of product will not be effective.', \\\"[substeps] Some reputable stores that specialize in this treatment are pharmacies, sports stores, drug stores, and cosmetic stores. Check your purchase and pharmacist's website to ensure you are not purchasing any pre-made product.\\\", 'Unfortunately, you can experience dizziness, nausea, and fainting. If you buy a product that contains scents that irritate your nose, such as benzoyl peroxide, retinol, or rubbing alcohol, consult your physician or dermatologist.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}