{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e78e6380-b929-5e3c-af34-798aa0ffce5d","task_key":"default--validation--23480","task_revision_id":"1","upstream_id":"23480","short_description":"HellaSwag validation 23480","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to dye your hair with washable markers [title] Choose your color (s). [step] Crayola washable markers work well and come in a variety of different colors and shades, but any brand of marker will do as long as it is labeled \\\" washable \\\". Once you've chosen your color (or colors), you need to get the ink out.\",\"endings\":\"['One solution is to have the tip of a finger hover over an ink pen. Write the ink color (s) on a paper plate in the color wheel.', 'With just a little bit of force, you can break open the marker. [substeps] Using scissors, pry the stopper off the end so that the back of the marker is completely open.', \\\"[substeps] Ink pens or a ink brush are preferred, as they provide easy picking up and flushing utensils and don't require water. Get the ink out of your hand first.\\\", \\\"Any marker that doesn't look how you want it to look can fall off and cause a serious stain, though. [substeps] You can purchase washable markers at any pharmacy or fabric store.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}