{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"17fffab5-465c-5b1c-8f4b-0fdaecd3933a","task_key":"default--validation--31394","task_revision_id":"1","upstream_id":"31394","short_description":"HellaSwag validation 31394","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to do easy nail art with a toothpick for beginners [title] Remove any old nail polish. [step] Dip a cotton ball in acetone nail polish remover and place it directly on any nails that have old nail polish. Wait a few seconds and rub off the remaining polish.\",\"endings\":\"['[title] Cut a rag swab with nail polish remover. [step] To make small pieces, cut one rag twice in one place.', '[substeps] Make sure to get all of the old coat off before you begin applying a clear base coat or base color. Old nail polish can prevent these new coats of base or paint from properly sticking to your nails.', 'Buff the old nail polish remover off with a clean cotton ball. [substeps] Work the fabric areas with acetone.', '[substeps] If any polish still remains on your nails from cleaning them with acetone, you can use a different cotton ball for this step. [title] Use the toothpick to sharpen your nail.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}