{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0d3399fc-31f8-5a6c-b4ea-7ef300570081","task_key":"default--validation--26652","task_revision_id":"1","upstream_id":"26652","short_description":"HellaSwag validation 26652","config":"default","split":"validation","body":"{\"activity_label\":\"High jump\",\"ctx\":\"The guy takes off running at an angle and is able to do a backwards jump over the six foot bar and clears it. after\",\"endings\":\"[', the guy is able to do the four un more flips over and over again.', 'he is done, the anoter basically gives the signal and walks up to the balance bar as the crowd cheers, claims their score and turns to cheering for a second.', ', he is able to perform another high jump and throws flapping ropes across the beam very far.', ', more athletes appear and they follow suit by doing high jumps and clearing the bar without scratching.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}