{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ac1429f0-68ae-53dc-9df4-8c17e1775bb8","task_key":"default--validation--23059","task_revision_id":"1","upstream_id":"23059","short_description":"HellaSwag validation 23059","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to wear metallic lip color [title] Apply your regular foundation first. [step] To start, you should apply your regular foundation. Dab the foundation on using a makeup brush or your fingers.\",\"endings\":\"[\\\"From there, mix the foundation in your face using your hands, a sponge, or a makeup brush. [substeps] If you don't normally wear foundation, you can skip this step.\\\", 'Working diagonally between your lips, apply powder foundation to foundation under your eyes and nose. Apply your regular foundation to your chin and cheeks.', 'Then, dab it on the center of your cheek where your center lip should be. [substeps] Your foundation should also be evenly applied to your cheeks, at the corners of your mouth, on either cheek, and on your chin.', \\\"Following the line you've drawn with the brush, wait for it to dry. However, if you're going for a classy, suave look, you can apply a bit of your favorite lipstick for added feel.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}