{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8681fdea-480e-5019-a3bd-0a0f3b35a5b1","task_key":"default--validation--5651","task_revision_id":"1","upstream_id":"5651","short_description":"HellaSwag validation 5651","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to achieve candy colored hair without a salon [title] The strand test : [step] Before dying your actual hair, you should follow these steps to make sure your dye will give you the color you want. [substeps] Cut a small piece of hair to test. Mix a teaspoon of bleach with a teaspoon of developer in your bowl.\",\"endings\":\"['Saturate hair sample with mixture check lightness after 20 minutes. Leave mixture on longer if necessary, but not longer than 45 minutes.', 'Read the instructions on the packaging to be sure the bleach will penetrate your hair just as deeply as the developer. When the developer is washed out, rinse your hair.', 'Rub your hair the color you wish to dye into an inconspicuous spot. Wait an hour or two for your color to change.', 'Dab the color on the piece of hair and let it dry completely. Wash the hair several times, but make sure to read product instructions and wash and condition the hair thoroughly after you add the dye.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}