{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3bb2e538-9b43-51ea-9c53-93e45b6a854c","task_key":"default--validation--9053","task_revision_id":"1","upstream_id":"9053","short_description":"HellaSwag validation 9053","config":"default","split":"validation","body":"{\"activity_label\":\"Long jump\",\"ctx\":\"A man performs a long jump on a track and into a sandpit in front of judges and onlookers on the sidelines. after the man\",\"endings\":\"['is shown he gets up from his seat and shakes hands with his name and friends, walks off stage where people applaud and wave him on stage and live photos appear.', 'on the track becomes a golden retriever.', 'falls down with two others cheering as he continues to jump on the track.', 'lands the jump he explodes in celebration by clenching his fists, after which a slow replay of the jump plays and a scoreboard displays his score.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}