{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0494e165-e2d7-56fe-98e8-1e5ffe41126c","task_key":"default--validation--31479","task_revision_id":"1","upstream_id":"31479","short_description":"HellaSwag validation 31479","config":"default","split":"validation","body":"{\"activity_label\":\"Making a cake\",\"ctx\":\"A woman stands on front a table with baby products, then the woman rolls up diapers and puts around a circular container with a ribbon. After, the woman puts diapers around a folded towel with a ribbon. next\",\"endings\":\"[', the woman put a pink bottle between the folds of the fold and put in a glass of water, after she fold the towel and put the liquid around the container.', ', the woman fills the circular container with baby products and put on top the towel, a plush and a bottle.', ', the woman put a blue rubber hose and put from the bottom of a diaper a blue rag with rolled up material and places tiny rocks on top of the soiled rag.', ', the woman put her hand in a paper receptacle and spreads the diapers near plastic buckets.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}