{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ccb15229-7c27-561a-aca7-7fe724506d7f","task_key":"default--validation--26248","task_revision_id":"1","upstream_id":"26248","short_description":"HellaSwag validation 26248","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to teach your kid to hold a pencil [title] Get the right pencil for your child. [step] Using short pencils, like golf pencils or normal pencils broken in half, can help children find the right grip on their own. Short pencils have less room for unnecessary fingers, so your child has little choice but to use the correct three-finger grip.\",\"endings\":\"['[substeps] You can purchase short pencils at most craft and office supply stores. [title] Help your child out with a pencil grip.', '[substeps] Good pencils to start with include the \\\" p \\\" and \\\" g \\\" pencils. Start in the bottom center pencil holder, and watch your child closely as he tries to hold the pencil in place.', '[substeps] For more information on how to handle a pencil, visit. [title] Differentiate between pencils that are powered by the hands and pencils that are used with both hands.', '[substeps] A finger with a longer pencil point allows children to hold pencils with much more control. A finger with lower points is available for trying to control just about any finger position.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}