{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c2de6ce2-55ef-5f7c-80a5-425be5be44b3","task_key":"default--validation--42094","task_revision_id":"1","upstream_id":"42094","short_description":"HellaSwag validation 42094","config":"default","split":"validation","body":"{\"activity_label\":\"Calf roping\",\"ctx\":\"People are in a rodeo arena while a man on a horse prepares a rope with a loop. then\",\"endings\":\"[', two men secure the rope by doing pulleys.', ', the man talks hands on race line while people walk in pass next to the horses.', ', the bull get chased by a crowd.', ', the man spins the loop and throw to catch a calf, then run and tie the calf.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}