{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4d96d5a1-7de5-5d83-b7b6-642088421ce4","task_key":"default--validation--10124","task_revision_id":"1","upstream_id":"10124","short_description":"HellaSwag validation 10124","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to treat feline retinal disease [title] Watch your cat's movements in dark areas. [step] The retina is made up of structures called photoreceptors-rods and cones-that allow your cat to see light and color. With pra, the rods and cones start breaking down.\",\"endings\":\"['Often, lumps or blobs are the result of an obsession with bright white centers from aging retinal skin. You may notice a particularly large pile of this fungus a few hours after a quit session.', 'Each class of mesh refracts light into one different area. The easiest way to rid your cat of photoreceptors is to place them in narrow, hiding spots, such as the casing housing of an outdoor refrigerator.', 'Without the rods, your cat would not be able to see very well in dark areas, like a hallway. You may notice your cat avoiding dark areas of your home.', \\\"This dislodges an existing layer of color from your cat's skin. Only a part of the brain can see this formation.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}