{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0647f735-fcfd-5ad2-9fac-bb424e79508f","task_key":"default--validation--42175","task_revision_id":"1","upstream_id":"42175","short_description":"HellaSwag validation 42175","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to make lemon bars [title] Preheat your oven to 350 ° f (175 ° c). [title] Prepare 9 by13 inch (22.86 by 33.02 centimeters) baking pan. [step] The simplest way to prepare one is to lightly grease it.\",\"endings\":\"[\\\"Since you'll be cooking the bars in the oven, you don't need to grease your pan yet. [substeps] Alternatively, bake the bars in your preheated oven.\\\", 'Use a rag or newspaper to line the pan with greased metal. [step] Mark where you want your bars to go and lay out a lightly greased palette.', 'If you plan on taking the bars out of the pan, line the pan with parchment paper instead be sure to leave some paper hanging over the sides. This way, you can use the excess paper like handles and lift the pastry out before you cut it.', 'You can grease the pan on the stovetop or inside with baking soda and cooking spray. Make sure to grease as evenly as possible so that the mixture will last much longer.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}