{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9b12ef5f-9924-59f3-80ac-8c7ab5573169","task_key":"default--validation--24538","task_revision_id":"1","upstream_id":"24538","short_description":"HellaSwag validation 24538","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to make hush puppies [title] Preheat a deep fryer to 365 ° f (185 ° c) or heat up 2 cups oil over medium-high heat. [step] The most common oil used is peanut, but vegetable, sunflower, or canola oil will do as well. As the oil warms to its appropriate heat, start mixing up ingredients.\",\"endings\":\"[\\\"[substeps] If you don't have a deep fryer, don't worry. You can use a piece of bread to check your oil's temperature as well.\\\", '[substeps] White flour, white sugar, baking powder, and potassium starch are common ingredients to start with. [title] Roast the puppies for 15 to 30 minutes.', 'Use a handheld mixer or a food processor to blend everything together in approximately three minutes. [substeps] If you have a deep fryer, you can skip this step.', '[title] Place the seeds in the middle of the oil. [step] Start by placing the seeds in the center of the oil and pressing down to avoid clumping the seeds together.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}