{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"11475a86-625d-53ee-8e1c-57ad0cb43e0f","task_key":"default--validation--10071","task_revision_id":"1","upstream_id":"10071","short_description":"HellaSwag validation 10071","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to do robin's egg nails [title] Apply a layer of base coat. [step] A clear base coat helps your nail polish go onto your nails smoother and brighter. Before you add any colored polish, slowly apply one layer of a clear base coat.\",\"endings\":\"['Let the base coat dry overnight before you move on. If you leave any polish on your nails, the color may not stay.', 'Take your second layer of base coat so that it covers the entire color of the first layer. [title] Polish the side of your nails, but remember to only let it touch the top part of your nails.', 'Allow your base coat to dry on your nails completely. [substeps] Opt for a fast dry base coat if you have one.', 'The coat should cover the entire nail, fully covering the nail. [substeps] A clear base coat will help you distinguish between the tones versus the other colors.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}