{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2633a450-3ac7-574c-91cb-ecd632566648","task_key":"default--validation--38252","task_revision_id":"1","upstream_id":"38252","short_description":"HellaSwag validation 38252","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to treat diarrhea in ferrets [title] Familiarize yourself with what is normal for your ferret. [step] A ferret's normal stool should be soft, but still formed (rather than runny). Check your ferret's cage or litter box from time to time and observe what its healthy stool looks like.\",\"endings\":\"['If you notice anything unusual, such as stools that are runny, green, or foul-smelling, contact your vet. [title] Take your ferret to the vet if you notice diarrhea.', \\\"Ferrets' stools shouldn't be watery, frothy, or thick. If they are, they may be sick.\\\", '[substeps] Healthy stools should be neutral color, i.e. solid green, and with no small flecks of green or other textures.', \\\"If you spot something like loose feces or urine, that's normal. The thinner your ferret's stool, the better.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}