{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4b743b72-48ea-58ae-a456-1f9d736b0da8","task_key":"default--validation--9677","task_revision_id":"1","upstream_id":"9677","short_description":"HellaSwag validation 9677","config":"default","split":"validation","body":"{\"activity_label\":\"High jump\",\"ctx\":\"An athletic young man is standing on a field and runs and jumps over a horizontal pole that is 5'6 high and lands onto the padded landing area below it and a slow replay of it plays. the man\",\"endings\":\"['is sitting in a tube on a flip board and holding onto something and doing flips and handstands on it behind him.', 'is shown kneeling down on the mat after completing the jump on his shoulders where he finally rolls to the mat and lands face down.', 'also slides backwards, performing balanced flips, and then lands with his feet located on the padded floor, and goes over the horizontal pole on any and all and lands after being done.', \\\"jumps again but this time at 5'10 high, he clears it, and once again it replays.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}