{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d8f6ac71-2060-5d98-a64e-8acbe736fd9d","task_key":"default--validation--37120","task_revision_id":"1","upstream_id":"37120","short_description":"HellaSwag validation 37120","config":"default","split":"validation","body":"{\"activity_label\":\"Washing face\",\"ctx\":\"A cat licking his right paw. Then starts to scratch his eyes with left paw and lick it for a while. eventually\",\"endings\":\"['moving his paws behind his ears licking and brushing repeatedly.', 'he bites it and stops him licking his paw.', 'he stops licking his paw, puts it down and gets his face disinfected with an adhesive solution.', \\\"some of the fur comes out and he gets blood on his paw, but he got to stop if he's frustrated, then he licks his paw again another time.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}