{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"41652cb1-bdb8-5934-aa27-257a11c8b5aa","task_key":"default--validation--24975","task_revision_id":"1","upstream_id":"24975","short_description":"HellaSwag validation 24975","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to apply fake nails [title] Take off your old polish and/or nails. [step] You'll need to take off your old polish, even if it's clear, in order to apply fake nails. Having old polish on your nails will make it harder for the fake nails to stick.\",\"endings\":\"['[title] File your nails into the correct shape using a pusher, as described in how-to guide. [step] A pusher looks like a stiff pencil on a flat flat surface.', '[substeps] Get rid of old polish before you apply fake nails. A good rule of thumb is that leave-in old polish on the pinkie, middle, and ring finger, and then remove your old polish, if any.', \\\"They'll end up falling off in a day or two unless you take this necessary step. [substeps] If you're already wearing fake nails, either acrylics or gels, go ahead and remove them.\\\", \\\"Remember that your fake nails were always for a person without old nail polish and they'll likely wear off when you take your fake nails off. [substeps] If you're wearing regular old nails, wash them just before applying fake nails.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}