{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6dc834a5-0cb0-5e7e-8a0e-ac9e5112dc0c","task_key":"default--validation--16709","task_revision_id":"1","upstream_id":"16709","short_description":"HellaSwag validation 16709","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to apply gel eyeliner [title] Choose the right product. [step] While it's not necessary to buy the most expensive cosmetics, read reviews about the product you're considering buying to find out whether it's worth it. [substeps] In addition to the liner, you will also need a brush that can create the look you want.\",\"endings\":\"['A thick brush will create thick, dark lines. A thin, angled brush will give you more control and allow you to make finer lines.', \\\"Brush brushes vary in quality, so ask to see a picture before jumping onto your first brush. It's difficult to see too much without tinting your lashes.\\\", \\\"Buy either gel eyeliner or your school's makeup products small brush. You can find these brushes online or at the makeup aisle in the cosmetics section of stores and drugstores.\\\", 'You can buy laser or gel eyeliner at a pharmacy. Gel eyeliner is solid stainless steel, which protects the eye from ultraviolet radiation.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}