{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b68a50fd-7542-563b-9a33-f5a5a5c9411a","task_key":"default--validation--5797","task_revision_id":"1","upstream_id":"5797","short_description":"HellaSwag validation 5797","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to cut a pixie cut [title] Start with damp hair that is already cut to at least shoulder-length. [step] Starting with hair that is already cut somewhat short will be a lot easier to work with than long hair. If the client's hair falls past their shoulders, gather their hair in your fist, then cut it off.\",\"endings\":\"[\\\"Don't pull back too much to make sure that it won't curl in a big, messy bunch. Wrap a piece of hair around the damp part or use a wide-tooth comb to smooth back any ends.\\\", '[title] Gather up the rest of your hair. [step] Trim the length that you gathered at their shoulder to leave a length of about half-an inch.', \\\"This is known as rough-cutting, and will make the length more manageable. [substeps] Don't worry if it looks uneven.\\\", 'Split their hair into even thirds, and tie in the bottom half of their hair. [title] Part the hair horizontally along the middle in a half-up v shape.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}