{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3888f582-196e-5b53-a898-5d4c7b5c8478","task_key":"default--validation--23604","task_revision_id":"1","upstream_id":"23604","short_description":"HellaSwag validation 23604","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to lighten lipstick with concealer [title] Take out your favorite shade of lipstick. [step] The best shades on which this method will work are red, wine, burgundy, deep brown, bright pink, purple, or orange. It is not recommended to try this with black or any other extremely dark lipstick in the blue or black family.\",\"endings\":\"['It contains too much dikiness and excessive darkness at the mouth. The idea is that if you spread the lipstick onto your lips, you will appear to be naked.', '[title] Take out your favorite shade of concealer. [step] It is best if it has a moisturizing texture.', 'It is unlikely that this will work well with blue or purple or burgundy. If you are having trouble seeing your lips in the best shade, look here.', 'Any of the shades you want to use will work, but have a friend drop by with your lipstick, so that you get the best results. [title] Apply concealer to the lipstick with a clean cotton or tissue or sponge.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}