{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bcfa76f8-43b1-5e94-bf46-673589a7e1f6","task_key":"default--validation--3300","task_revision_id":"1","upstream_id":"3300","short_description":"HellaSwag validation 3300","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to do a laced ponytail [title] Pull your hair up into a high ponytail. [step] Wrap a hair tie around your wrist and pull your hair up into a high ponytail. Hold your hair with the hand that has the hair tie on it.\",\"endings\":\"['The ponytail should be high on your head. [substeps] Braid all of your hair into the ponytail and secure the loose ends with bobby pins.', 'Insert your index and middle fingers into the ponytail until the hair tie reaches your wrist. Twist your wrist a few times to tighten it into the ponytail.', 'Use your other hand to slip the hair tie off and wrap it 2 to 3 times around your ponytail. [substeps] Use a brush or comb to get it as smooth as possible.', 'Use your hand to secure the ponytail just to the crown of your head. [substeps] Hair ties work well with almost any hairstyle except for pulled up ponytails, since they restrict the release of spikes.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}