{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"71d92ef2-c6c8-5d3f-a32d-2b0d251fec0e","task_key":"default--validation--6939","task_revision_id":"1","upstream_id":"6939","short_description":"HellaSwag validation 6939","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make gel eyeliner [title] Place a small amount of loose eyeshadow in a container. [step] To mix up your own gel liner in a hurry, find a loose eyeshadow pigment in the shade that you want to line your eyes with. Sprinkle a small amount into the lid of its container or on another flat surface that you can mix on.\",\"endings\":\"[\\\"[substeps] It's best not to use eyeshadow pigment with glitter. The particles may fall in your eyes if you make a liner with the glitter.\\\", \\\"[substeps] The eyeshadow will likely puff up and then dart back out in the center when you're finished. During this process, you may find that very small amounts result in a difficult looking look.\\\", 'Keep your tip brush or well-rounded brush handy; by pressing down on the rosebud eyeshadow, it will form a lengthening trace on the liner. [title] Line your upper and lower lash line.', 'Spread this into your eye as close to the lash line as possible with an eyeliner brush. [substeps] Generally, when you are applying gel eyeliner to your eyes, the lashes stick out less than the rest of your eyeliner.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}