{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b85e7ac0-f5a7-506f-9a0f-418d4b992c4e","task_key":"default--validation--22070","task_revision_id":"1","upstream_id":"22070","short_description":"HellaSwag validation 22070","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to remove set in stains [title] Purchase an oxygen bleach product. [step] There are many to choose from, but oxiclean is probably the most easily recognized. The specific product you buy is your preference, but you should look for a powdered formula that contains sodium percarbonate and sodium carbonate.\",\"endings\":\"[\\\"You will need at least four gallons of oxygen bleach per gallon to use your child's bathroom, although higher volumes of oxygen will generally be used if you have more children. Buy a suitable formula from a hardware store or online.\\\", \\\"At minimum, you should buy the product with 1% sodium percarbonate. A powdered formula does the same thing, but you don't have to worry about the chemical reaction going on too much.\\\", \\\"Other fillers will reduce the efficacy of the product. [substeps] Some other recommended brands are biokleen's oxygen bleach plus, oxo-brite (which is supposed to be earth-friendly), and vaska's oxygen brightener.\\\", 'This will help keep the stain away from you and keep it from infecting the entire garment. [title] Start with a clean cloth and a bit of bleach.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}