{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"21aaa199-1cb8-54c3-832f-09606d07cbae","task_key":"default--validation--46478","task_revision_id":"1","upstream_id":"46478","short_description":"HellaSwag validation 46478","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to add nut and seed butters to your diet [title] Select high-quality peanut butter. [step] You may be overwhelmed when selecting a peanut butter, since there are so many options. Read the ingredient label and avoid peanut butters that have added oils (especially hydrogenated oils) and sugars.\",\"endings\":\"['Smooth peanut butter is ideal because it has a lot of natural sugars and removes the fat. Even better, it will have a sweet smell that is pleasing to mouth-to-mouth.', 'To avoid and limit the risk of creating pitfalls, pick peanut butter containing added oils such as avocado, walnuts, sunflower seeds, fatty fish, and hemp seeds. [title] Sprinkle the peanut butter on your toast.', 'Opt for peanut butter that has no added fat. It is highly recommended that peanut butter be on the thick side as peanut butter already ages is easily accessible from the back of the package and should be easy to handle.', 'Try to buy a natural peanut butter that just contains peanuts (and possibly salt). Peanut butter is a great source of protein.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}