{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ee710713-e45d-52dc-9c4b-995ff95dcf43","task_key":"default--validation--26651","task_revision_id":"1","upstream_id":"26651","short_description":"HellaSwag validation 26651","config":"default","split":"validation","body":"{\"activity_label\":\"High jump\",\"ctx\":\"A black male athlete is outside in a field at a track meet preparing to clear a high jump. the guy\",\"endings\":\"['then gets in his long jump while 10 men watch on the sideline.', 'in the black shirt is competing against another man, who is running towards the high jump stage in his left hand.', 'takes off running at an angle and is able to do a backwards jump over the six foot bar and clears it.', 'then legs and abs the high jump and is announcin a little bigger then the past two which are both real.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}