{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2e093949-cbc8-5835-a3ac-ec51d9d42edd","task_key":"default--validation--21043","task_revision_id":"1","upstream_id":"21043","short_description":"HellaSwag validation 21043","config":"default","split":"validation","body":"{\"activity_label\":\"Having an ice cream\",\"ctx\":\"The lady hands the ice cream to the lady next to her and feed the baby something else. The baby reaches for the food. The baby reaches for the food again. the lady\",\"endings\":\"['wipes the babies face and stands her up.', 'help the dog to reach food.', 'picks the food up and takes a bite and cheers.', 'picks up the ice cream and feeds the baby snow cream.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}