{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"46519b9b-f714-5e2d-9d40-9fb58ccdf593","task_key":"default--validation--49710","task_revision_id":"1","upstream_id":"49710","short_description":"HellaSwag validation 49710","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to do negative space nail art [title] Begin with clear nails. [step] Remember, you're using the negative space of your nails to create art for this look. Therefore, your nails should be clear going into the process.\",\"endings\":\"['Open up a coat of nail varnish on the wedge tips and hold this for a couple minutes. Use a new coat to fill in the dark edges.', 'If you have any nail polish or nail polish residue, remove it with nail polish remover before you begin. [title] Place stripping tape horizontally across your pinky nail.', 'Each set of nails should measure one and a half in. This will be the number you use per nail.', 'Start by washing your nails so that all of the dirt is removed from your nails. Check with your nail art shop for permission to fill in your nails.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}