{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"57b0772a-b748-5272-82b7-593e36caf833","task_key":"default--validation--23990","task_revision_id":"1","upstream_id":"23990","short_description":"HellaSwag validation 23990","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to love cats [title] Don't neglect your pet cat or cats. [step] Cats will usually let you know when the want your attention or just your company. They don't have feelings exactly like us, but a neglected cat is never a happy cat.\",\"endings\":\"[\\\"Think about why this is and when you know your cat likes you. If you don't know and are unsure about your cat's ego, own it.\\\", 'A neglected pet needs your attention as much as a healthy, loving one. Animal shelters provide care to feral or live-in cats.', \\\"Don't be discouraged if you didn't get the cat you really wanted or if it is feeling ignored or neglected. [substeps] Don't nag unless necessary.\\\", \\\"[title] Don't be afraid of cats, cats are unique creatures. [step] Pet and stroke cats to them feel secure.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}