{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"abf5ebaf-8001-506e-a449-3510d03c9466","task_key":"default--validation--21837","task_revision_id":"1","upstream_id":"21837","short_description":"HellaSwag validation 21837","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to make green beer without food coloring [title] Get your hands on some light beer. [step] Pale ales, light lagers, and wheat beers do just fine; the lighter the better! You want the coloring to show up as best as possible. [title] Try adding spirulina to your beer.\",\"endings\":\"['[step] Spirulina is a type of vinaigrette, meaning it replaces unpeeled potatoes with mushrooms. However, it can also be a bit more bitter, so try adding spirulina in moderation.', \\\"[step] Spirulina is an ocean white frothy beer and doesn't have any charge. Spirulina tastes great everywhere, so make sure you obtain enough phyff to cover the ingredients on the inside of your beer.\\\", '[step] Spirulina is a blue-green algae that is incredibly high in protein and has a vibrant green color. Not only will adding spirulina to your beer help you achieve that lovely green beer color, it will also be great for your health.', '[step] Spirulina is found mainly on beer with vegetables and other fruits. Spirulina makes an excellent meal to try eating and leaves your body feeling full without having to taste the beer yet again.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}