{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c0d4bafb-2c20-5fdd-8cf3-af7b5e4e8327","task_key":"default--validation--13901","task_revision_id":"1","upstream_id":"13901","short_description":"HellaSwag validation 13901","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to pickle onions [title] Halve, peel, and slice the onion. [step] Red onions go great for this recipe; with their distinctive taste and deep red color, they actually turn pink after a couple days in the refrigerator. [substeps] You can slice the onions to the thickness you prefer-however thick, they'll still take the pickling solution.\",\"endings\":\"['[title] Mix the pickling liquid with the onions. [step] Add a handful of loosely chopped onion , 1 teaspoon pickling mix , 2 teaspoons freshly grated paprika, and ¼ cup of water to taste.', \\\"Don't worry if the onions look bigger or smaller along the sides and inside the pot, they can stand you fine. It's up to you if you'd like to separate them properly.\\\", 'Some people prefer to cut the onions in a full moon, which makes them easy to grab with a fork; others prefer them smaller, cut them into half moons. [title] Heat some water over the stove or in a kettle.', \\\"Two varieties of onions are excellent for this recipe; if you're not quite sure which kind of onion to use, read the ingredients, but simply pick the slightly larger version of the onion anyway. [title] Prepare the pickling solution and cut each onion into quarters.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}