{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e07f0d07-8fd3-53f8-98fb-5e2a216bc652","task_key":"default--validation--30391","task_revision_id":"1","upstream_id":"30391","short_description":"HellaSwag validation 30391","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to dye your hair with lemon juice [title] Squeeze the juice from three fresh lemons. [step] Cut each lemon in half, then remove the seeds with the tip of the knife. Squeeze each half into a bowl or measuring cup.\",\"endings\":\"[\\\"One lemon is good to begin with, as it is thinner and less acidic than the other two lemons. [substeps] It's a good idea to apply the lemon juice to your hair while it's still damp, if you need to add it later.\\\", 'You may need slightly more or less, depending on the length of your hair. [title] Add two parts lemon juice and one part leave-in conditioner to a spray bottle.', \\\"How many lemons you use depends on how thick you want the dye to be. [substeps] Lemon juice takes longer to dry than non-lemon juice, so you can use as much or as little lemon juice as you'd like.\\\", '[substeps] Some people might prefer that you use a knife rather than a spoon. This allows you to apply the lemon juice without damaging the delicate roots of your hair, and also adds a nice color to your dyed hair.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}