{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"79b7a79b-afb2-574f-8ac9-0ec83ab17811","task_key":"default--validation--23332","task_revision_id":"1","upstream_id":"23332","short_description":"HellaSwag validation 23332","config":"default","split":"validation","body":"{\"activity_label\":\"Cutting the grass\",\"ctx\":\"A boy in young boy drives a small tractor lawn mower around the yard of a house maneuvering around obstacles. the boy\",\"endings\":\"['mows over some small flowers in the grass.', 'walks in a circle the approached the camera, and then went back the same way he came from.', 'stops then passes the tractor.', 'pushes the edge of the mower up and down along the lawn path as leaves begin to fall on the lawn.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}