{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f167a4d3-05cb-5951-8d65-6ebdac4f8146","task_key":"default--validation--5181","task_revision_id":"1","upstream_id":"5181","short_description":"HellaSwag validation 5181","config":"default","split":"validation","body":"{\"activity_label\":\"High jump\",\"ctx\":\"The guy seems to be practicing how high he can jump by jumping over the pole many times, each time a person raises the pole higher and higher to see if the guy can jump over it. one jump the guy\",\"endings\":\"['does but the pole falls then he jumps but he fails to land very far.', \\\"ensure and then jumps over the pole and does somersaults once he's standing directly over it.\\\", 'knocked the pole down when he tried to hump over it.', 'jumps from and lands parallel with the pole pole.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}