{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7b172c72-8ea7-5038-9511-63535643d34c","task_key":"default--validation--40557","task_revision_id":"1","upstream_id":"40557","short_description":"HellaSwag validation 40557","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to use powder hair dye [title] Complete a skin allergy test 48 hours beforehand. [step] Although powder hair dye is usually more skin-friendly and chemical-free, some people may experience allergic reactions. If this is your first experience with a certain brand, mix a small amount of the powder with water and apply it to your skin.\",\"endings\":\"['This test will allow you to see if any latex reacts with any pigment in your skin or hair. [substeps] Acids (including lanolin and iodine) may cause skin irritation.', 'Watch out for slight allergic reactions as the powder will infuse the hair with dye. [substeps] If your skin reaction is still mild, you may skip this step.', '[title] Wait for an hour for the dye to react with the water. [step] Due to the size of your dye machine, it may take a longer time for the dye to respond to the water.', 'Once two days have passed without red or irritated skin, you can apply the dye knowing that you are not allergic. [substeps] Common allergy signs may include: intense stinging, skin rash, burning sensations, swelling, or blisters.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}