{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"59e55917-0c66-5992-8e0f-d61231db629b","task_key":"default--validation--12878","task_revision_id":"1","upstream_id":"12878","short_description":"HellaSwag validation 12878","config":"default","split":"validation","body":"{\"activity_label\":\"Hurling\",\"ctx\":\"Another play starts and this time another person is ran into except it is more fragrant and the person lays on the ground hurt for quite some time. after\",\"endings\":\"['that another guy plugs into the playmate.', ', the third time the person belongs is picked up and dropped off then he moves to another area to play again.', ', the second set of lean singers is played and the crowd is now louder than when before.', 'consulting with the referee and seeking help, the male finally gets up and the rest of the game continues.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}