{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3cfccda9-5b6f-5121-b978-6c61c27abad8","task_key":"default--validation--34030","task_revision_id":"1","upstream_id":"34030","short_description":"HellaSwag validation 34030","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to make chewing gum [title] Heat the ingredients. [step] Place the gum base, corn syrup, glycerine, citric acid and bubble gum flavoring in the top part of a double boiler. Place the double boiler on the stove and turn the burner to medium high.\",\"endings\":\"[\\\"[title] Add the melted gum base mixture to the warm water. [step] Beat the ingredients together one at a time until they're the consistency of peanut butter.\\\", 'Put the rest of the ingredients except for the sugar back into the boiler and heat them on medium until the sugar is dissolved. [substeps] Stir the gum base often to keep it from scorching.', 'Stirring occasionally with a spoon, heat the mixture until it becomes warm and gooey. [substeps] You can find gum base, glycerine and citric acid at candy supply stores or online retailers.', \\\"[substeps] There are many ways to do this, but you want to choose a place that is relatively safe, away from heat sources. If you're working in a hot environment, place the ingredients in a glass bowl or bowl and let the ingredients melt for 20 to 30 minutes before proceeding to the next step.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}