{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"39a58c33-18a5-5cbe-b43a-a4006ae76669","task_key":"default--validation--10739","task_revision_id":"1","upstream_id":"10739","short_description":"HellaSwag validation 10739","config":"default","split":"validation","body":"{\"activity_label\":\"Calf roping\",\"ctx\":\"The view changes to show a group of men with their horses and they're each holding onto their ropes and doing various things with them like tying and swinging them. the focus\",\"endings\":\"['switches to a man talking whose name is trevor brazile and it flashes from him talking and then to clips of him riding his horse in different arenas as he is also wrangling a small calf.', \\\"turns back to the horse standing in the warm weather and he is commentating on what he's doing.\\\", 'is now back on the contestants and they are green with sweat and a lot of other people are doing the same thing.', 'changes back to the men on the sides of the ring to try to coach the horses on what to do.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}