{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1de751f2-8d32-5b2b-8ad6-6c6c96411f5d","task_key":"default--validation--41103","task_revision_id":"1","upstream_id":"41103","short_description":"HellaSwag validation 41103","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to clean canvas sneakers [title] Gather the cleaning gear. [step] You will need nail polish remover and a cotton ball, q-tip, cotton bud, or cotton pad. [title] Add some nail polish remover to the stain on the canvas sneaker.\",\"endings\":\"['[step] You may use some of the following diy markers : [substeps] Mash 5 ml of remover with water in a bowl then mash some lemon juice with a sponge. Rub the cleaning scum on the canvas sneaker.', '[step] Before you apply the stain, fill the shoes with warm water and a drop of dish detergent. Then, dab on a drop of warm water.', '[step] Use small amounts of nail polish remover each time, as some fabrics and materials react to certain amounts of nail polish remover due to the acetone it contains. You may wish to do a test area first.', '[step] Spray the stained shoe with an acetone based cleaner. [substeps] Do not apply concealer directly to the stain; you want to let it soak in and absorb the acetone.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}