{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"61a3f1b2-02b3-553e-b810-f270df87b978","task_key":"default--validation--4518","task_revision_id":"1","upstream_id":"4518","short_description":"HellaSwag validation 4518","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to have a gluten free thanksgiving [title] Offer to cook the thanksgiving dinner yourself. [step] This is the only surefire way to ensure that your thanksgiving will be both gluten-free and filled with tasty menu items that you will enjoy. You will run the show, which means that you can filter what comes into the kitchen, and what makes it onto the dining room table.\",\"endings\":\"[\\\"[substeps] If you will be a guest at someone else's home for thanksgiving, speak with the host about your gluten sensitivity ahead of time. Offer to make some gluten-free side dishes yourself.\\\", \\\"Don't be afraid to make it yourself, though. Some good choices could be a mixed fruit cocktail, a wine and salad topped with parsley, or a mixed beet salad.\\\", 'Besides, the time we spend prepping the meal also counts as preparation, as it will interrupt the recipe preparation process. [substeps] Set up the thanksgiving dinner at a restaurant that specializes in gluten-free foods.', \\\"Create a basic meal plan for what you'll make for the thanksgiving feast. [substeps] This will not make all the food but it will let you pack your thanksgiving stir-fry appetites.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}