{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"94edf700-13ad-5369-b0a8-cf6282594b6a","task_key":"default--validation--44832","task_revision_id":"1","upstream_id":"44832","short_description":"HellaSwag validation 44832","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make ice balls [title] Buy a package of water balloons. [step] This method is cheap, quick, and easy-perfect if you don't want to spend money on specialty molds for making ice balls. All you'll need is a few round water balloons (and, of course, water and a freezer.\",\"endings\":\"[') [substeps] Uninflated balloons: this method requires 2 balls , 1 ice cream stick and 2 water balloons in a large bag (1 at a time). Open the sides of your volcano and shake the tupperware from side to side a few times.', \\\") [substeps] Of course, there is no telling if the water balloons will be successful; you just know that they will eventually explode on the first try. Plus, they're basically too small to correctly hang from your roof.\\\", \\\") [substeps] Remember that, if you're using your ice balls to cool your drinks, they'll need to be able to fit inside your glasses. Thus, when it comes to water balloons, generally, the smaller, the better.\\\", ') [substeps] Use unopened water balloons, because they tend to be too full. Plastic ones are dry and will be harder to put in the freezer.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}