{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4f33d5ca-73aa-546f-a5f9-fddd6b7b0e46","task_key":"default--validation--29694","task_revision_id":"1","upstream_id":"29694","short_description":"HellaSwag validation 29694","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to apply pancake makeup [title] Wet your sponge. [step] Pancake makeup needs to be activated with water, and should be applied with a dense sponge. The great thing about pancake makeup is that you can control the amount of coverage you get by adding more or less water (less water means less diluted makeup and heavier coverage).\",\"endings\":\"[\\\"[substeps] For regular coverage, soak your sponge with water and then squeeze out the excess so the sponge is damp. For lighter coverage, soak the sponge and then give it a gentle squeeze so it's not dripping wet.\\\", \\\"This means your makeup will be waterproof for a while, which means the matte makeup is the most consistent when you apply it. [substeps] If you don't have a sponge, you can just use a wet sponge.\\\", \\\"[substeps] When wetting the sponge, just blot the butter where you wish your pancake makeup to be. It's usually easiest to apply with the sponge when wiping your skin clean, as you won't have to worry about mistakes in little areas.\\\", '[substeps] You can squeeze the sponge directly into the butter, juice, or other container with the appropriate amount of pressure. Alternatively, you can use a bowl or dish large enough to hold the makeup of your choice and mix and stir the two ingredients together.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}