{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"75e6ec91-faf1-5ff7-bf59-77d258ee5eae","task_key":"default--validation--37309","task_revision_id":"1","upstream_id":"37309","short_description":"HellaSwag validation 37309","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make a chocolate bubble bath [title] Buy unscented bubble bath. [step] You will need to buy bubble bath to create the bubbly, foamy part of the bath. The chocolate will provide the scent and color of the bath, so do not buy a scented bubble bath.\",\"endings\":\"['Consider buying a bubble bath that is scented with essential oils, natural ingredients, or herbs and that are unscented. [substeps] Read the products you buy and make sure to read the ingredients on the soap bottle.', 'Instead, look for a bubble bath with as few ingredients as possible. You can find bubble bath at many supermarkets and bath and body stores.', '[substeps] You can also try mixing the bubble bath with a few drops of essential oils to create a unique scent. [title] Fill a large glass or plastic bowl with bubble bath-sized scoops.', 'Buy one in a gallon jug, as opposed to a standard glass bottle. [substeps] Avoid using bubble bath products, chocolates, or other exotic (carbonated) products.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}