{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5d7763c4-b060-56c9-84bb-13e254073ffe","task_key":"default--validation--22385","task_revision_id":"1","upstream_id":"22385","short_description":"HellaSwag validation 22385","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to turn mashed potatoes into gnocchi [title] Measure out the mashed potatoes. [step] You'll need 2 cups (650 g) of prepared mashed potatoes. You'll have an easier time rolling out the gnocchi dough if the mashed potatoes are at room temperature.\",\"endings\":\"['You should also run the mashed potatoes through a strainer if they seem lumpy. [substeps] Avoid using mashed potatoes that have bits of the potato peel in them.', 'For best results, cover and chill the mashed potatoes for at least 1 hour to soak up any excess water before rolling them. [substeps] Be especially careful to leave no chunks of dough separating from the mashed potatoes.', \\\"Move onto the next step after measuring out the mashed potatoes, unless you're prepping them for use as dough. [title] Press the bottom of a heavy-bottomed bowl into the mashed potatoes.\\\", '[title] Measure out the yeast and the seasonings. [step] Turn a medium mixing bowl over and pour the yeast and 2 tablespoons (45 ml) of water into the bowl.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}