{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"066fc3a0-2135-5264-adc3-b9b7a9987e84","task_key":"default--validation--19373","task_revision_id":"1","upstream_id":"19373","short_description":"HellaSwag validation 19373","config":"default","split":"validation","body":"{\"activity_label\":\"Hitting a pinata\",\"ctx\":\"A man spins in circles to make himself dizzy and disoriented. He then swings a bat at a pinata, missing and hitting the woman behind him in the head. she\",\"endings\":\"['reels and falls to the ground, and everyone walks away.', 'drops to the ground in pain.', 'tries again, squeezing the bat and hitting the pinata still.', 'kicks him back at the pinata, although she is unable to hit it.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}