{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"25d2f290-8df4-5169-82d3-8bb523f4cf3d","task_key":"default--validation--49493","task_revision_id":"1","upstream_id":"49493","short_description":"HellaSwag validation 49493","config":"default","split":"validation","body":"{\"activity_label\":\"Slacklining\",\"ctx\":\"A strong young man is shown on the other side of the tree pulling the line to ensure that is the tightest it can be around the trunk of the trees. Then two couples appear and the men are assisting females tight rope across the slack line. next\",\"endings\":\"[', more couples appear and the who are there have a harder time doing much of anything than pulling the rope together while some people in the background watch, some kind of various activities hold in store.', ', a stretching team then appears that person 87 or 23 different children moving back and forth across the rope over tight lines and trying to make it across without falling and injuring one another.', ', one of the couples finishes up and the name of her blood type flashes across the screen briefly.', ', the two men begin to take their turn and walk across it jokingly holding up bottles of wine.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}