{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"019ffa3b-9e50-5f9e-97b6-5443190042b1","task_key":"default--validation--15302","task_revision_id":"1","upstream_id":"15302","short_description":"HellaSwag validation 15302","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to burn letters into wood [title] Pick out wood. [step] Any wood surface can be. However, some are better than others.\",\"endings\":\"['Do some research about the different types of wood you have so you can determine which material will work best for your project. [substeps] Make sure the wind in your area and temperatures are acceptable before roasting your letters.', \\\"Light-colored and soft wood, such as bass, works especially well. This is because the wood burning stands out on the light color and you don't have to push very hard to make your marks.\\\", \\\"Any surface that is more akin to wood (like cobblestone, concrete, or soda is for instance) works. [substeps] Look at the grain of the wood, and if you can't find that grain, use a compass or a straightedge to find a grain grain.\\\", \\\"The first step to burning wood is weighing it down in the right place. You want to make sure you have something either loose on the surface, or that doesn't suck up very much of the wood, especially a pen or pencil.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}