{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6700fc4b-2d4a-5376-922e-29b878c234ad","task_key":"default--validation--4431","task_revision_id":"1","upstream_id":"4431","short_description":"HellaSwag validation 4431","config":"default","split":"validation","body":"{\"activity_label\":\"Horseback riding\",\"ctx\":\"The woman leads them into the barn and she puts a helmet on the little girls head, they walk to get horses and at some point the duggar mom put a helmet on as well. in\",\"endings\":\"['the end the man jumps on the horse and ties him up and walks off to finish something.', 'between interviews they show the duggar mom and the little girl getting instructions about their horse and then getting on their horses and riding on them in an enclosed area.', 'the barn they take their helmets off and they throw them down and take off riding the horses through the snow.', \\\"the end the joke is on them while they're riding their horses and it is said that old father is the strongest horse.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}