{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3e7cf772-74d0-5196-a731-b5dc2e46ffa5","task_key":"default--validation--10742","task_revision_id":"1","upstream_id":"10742","short_description":"HellaSwag validation 10742","config":"default","split":"validation","body":"{\"activity_label\":\"Calf roping\",\"ctx\":\"Clips of both of men talking and clips of them riding horses and wrangling calves play and rotate between the two of them. the outro video\",\"endings\":\"['finally comes through and it shows both men close up outside bouncing around.', 'begins and it shows a hanging banner at the arena that reads \\\" calgary stampede \\\", a woman riding a horse and holding a red flag that say\\\\'s \\\" 1912-2012 calgary stampede \\\", and a calf is running across a dirt field.', 'ends with video credits as well as the change of language shown from the start of the video.', 'logo appears with infographic and blue paragraph saying down the line, \\\" viewers please find a property that would be willing to set up an interview with both of them, separately, by email, via texts and phone \\\" and pauses to let the others catch up and continues talking into the mic.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}