{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ad68b492-3e61-5bdb-8a4e-8e9bab79993a","task_key":"default--validation--16839","task_revision_id":"1","upstream_id":"16839","short_description":"HellaSwag validation 16839","config":"default","split":"validation","body":"{\"activity_label\":\"Horseback riding\",\"ctx\":\"This woman rides the white/brown horse under the shady trees for a little bit. The horse is nodding its head and walks in circles a few times. in the end\",\"endings\":\"['the girl stands next to the pony and talks to the family and the little child.', 'it stops and it is said out loud to the camera and attachment.', 'the person win and they shake hands with the winner of the rodeo, the win received her trophy and walked away shaking hands with her cameraman.', ', the horse stops walking and just stands in place while the woman is still sitting on the saddle.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}