{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bc63c6d9-66cb-59c4-afcc-9483f28dfb40","task_key":"default--validation--18541","task_revision_id":"1","upstream_id":"18541","short_description":"HellaSwag validation 18541","config":"default","split":"validation","body":"{\"activity_label\":\"Cutting the grass\",\"ctx\":\"A boy with dark hair and no shirt is mowing a dirt patch with small patches of grass using a large, orange, push lawn mower. as the boy\",\"endings\":\"['continues, he starts to try and mow the grass with the yellow, wind powered, when he is done, he tosses the mower and walks onto the right side of the yard to break the grass apart.', 'finishes his task, he takes the mower off of the camera frame but instead of disappearing behind the screen, he appears to be zooming in on the grass on the left.', 'mows, he pauses and flashes to the camera and two ends are shown before suddenly zipping back to the beginning.', 'mows the dirt patch, large amounts of dirt come up from the ground in a smog like blizzard.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}