{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"18e1d820-bc7f-5163-b8e0-e993d3c8911a","task_key":"default--validation--20192","task_revision_id":"1","upstream_id":"20192","short_description":"HellaSwag validation 20192","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make eyeliner wings even [title] Cut off two small pieces of tape. [step] You can use household scotch tape, or you can use surgical tape if your skin is a bit more sensitive. You will need one small piece for each eye.\",\"endings\":\"['You can always use 24 to 48 segments of duct tape if you want to make more liner. [substeps] Keep in mind that your wings might start to go out of style once the tape has dried.', '[title] Use your tweezers to prick each eye in the \\\" c \\\" shape using the scratch tool. [step] Each eye needs to be in roughly the same shape, preferably with parts without movement.', '[title] Make two smaller butterfly-shaped lines with the tape : [step] You can make a smaller butterfly-shaped wing by cutting it in half. Before you brush it over your face, wind it around your nostrils to make one side longer.', 'Fold one corner of the tape down, so that it creates a little flap that will make it easier to peel the tape off of your skin. [title] Place the tape on the corners of your eyes.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}