{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5f08e8b4-ba04-56fa-ba5b-29007fba826a","task_key":"default--validation--37556","task_revision_id":"1","upstream_id":"37556","short_description":"HellaSwag validation 37556","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to clean your hair with beer [title] De-carbonate your beer. [step] Pour a bottle or can of beer into a jug or bowl and let it sit overnight or throughout the day. The more air surface your beer is exposed to, the faster it will oxidize and become flat.\",\"endings\":\"['Rinse it off in the morning, and then use it to clean your hair again. [substeps] Keep in mind that beer often contains chlorine, which should be avoided while dealing with your hair.', 'If you drink your beer through coffee or tea, avoid standing in the tap during the entire day. Pour it directly into the coffee or tea cooker.', \\\"[substeps] Beer becomes flat by releasing the carbon dioxide it creates. It's important that the carbon dioxide is gone from the beer when using it to treat hair as carbon dioxide combined with water can create hard water or water with a high mineral content.\\\", '[substeps] The best time to de-carbonate your beer is before noon. This will ensure that the beer is not going to harden before a full day passes without exposure to sunlight.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}