{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"36c389a3-2561-5303-8e55-5e645c58a496","task_key":"default--validation--46921","task_revision_id":"1","upstream_id":"46921","short_description":"HellaSwag validation 46921","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to make cookie butters [title] Select a crumbly cookie. [step] Part of the beauty of cookie butter is that you can make it with practically any kind of cookie. That being said, you should think about whether your favorite cookie is ideal for turning into a spreadable substance.\",\"endings\":\"['It can be wet, crumbly, or very crumbly. [substeps] Choose whichever one you like, as long as the texture suits it.', 'Once you have selected your crumbly cookie, you want to make it easier to chew and apply to your mouth and enjoyment. [title] Drizzle melted butter over the surface of the cookie.', \\\"[substeps] You can cut a cookie in a large bowl so that it is not too thin. In addition, if you are making butter that is thin but smooth, then it will be easier for the butter to mix in because you won't need to butter the bottom of the bread or biscuit.\\\", 'You want a variety that crumbles into coarse granules easily, rather than a chewy recipe which will just get mushy or pasty when blended. [substeps] For example, snickerdoodles, sugar cookies, pecan sandies, and oatmeal cookies all make great selections for cookie butter.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}