{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1960d859-5ab7-5e91-9c23-b4e0596448d7","task_key":"default--validation--35466","task_revision_id":"1","upstream_id":"35466","short_description":"HellaSwag validation 35466","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to diagnose entropion in boxers [title] Look for squinting. [step] If your dog is feeling uncomfortable, you may notice him squinting or narrowing his eye. Compare one eye with the other to see if one eye is worse.\",\"endings\":\"['If one eye is worse than the other, then the vet might examine the eye rather than going any further than the other to rule out entropion. The vet may want to do a rectal biopsy if they suspect entropion.', \\\"If your dog's eyes are dull or faded, that is a development of entropion. [substeps] Entropion may also be an early sign of obesity or alzheimer's disease.\\\", 'Although entropion will look worse than any other, any eye that is unfocused, glazed, or inflamed is probably entropion. It will also look and feel different when closed.', 'If both eyes are affected then the dog may appear to have narrow eyes. [substeps] If the eye is really bothering your dog, he may even hold his eye shut to keep the lid from rubbing against the eyeball.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}