{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f7fa96de-cc85-55c5-8d67-54666b5430eb","task_key":"default--validation--45043--variant-d7790bf0198b","task_revision_id":"1","upstream_id":"45043--variant-d7790bf0198b","short_description":"HellaSwag validation 45043","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to make crostini [title] Turn your oven onto broil, and remove a cookie sheet to broil on. [step] Line this cookie sheet with aluminum foil or with parchment paper to help ease the cleanup process. [title] Melt together the butter and olive oil.\",\"endings\":\"['[step] In a small sauce pan, melt 2 tablespoons of butter and 1 tablespoon of olive oil until well-combined. Cook over a low heat until the butter melts.', '[step] Pour 1 tablespoon (15 ml) of butter and 2 tablespoons (45 ml) of olive oil in a small saucepan over medium-high heat. Once it starts burning, add in the dry ingredients to soften them.', '[title] Add in the garlic, and any other seasoning as desired (no more than a dash of salt, and a shake or two of pepper though). [title] Slice the loaf into thin slices (around 1 cm in thickness).', '[step] Turn the burner on low, and add the butter in little increments or in small quantities of whatever you prefer, until the butter and olive oil are heated fully. [title] Toss in the flour, sugar, almond, and salt.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}