{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0fd23dc3-e13a-52f3-ad48-4b73f508d8a9","task_key":"default--validation--23904","task_revision_id":"1","upstream_id":"23904","short_description":"HellaSwag validation 23904","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to make a chocolate bouquet [title] Gather some hershey's kisses. [step] You will need two hershey's kisses for each rose. Your bouquet can have as many \\\" roses \\\" in it as you want.\",\"endings\":\"[\\\"You can even make more than one with equal parts of white chocolate and milk chocolate. Read the instructions that come with the hershey's kiss kit to get ideas for how many to use.\\\", \\\"You'll either want to get skinny, try substituting rose mix with different ones on the spot, or you can use chocolate syrup, etc. [substeps] You can buy flavored hershey's kisses if your roses are in a lot of colors, making them ideal for mixing with chocolate syrup.\\\", \\\"Your hershey's kisses can be in different colors (olive or chocolate are common red colors) and you can make multiple bouquets from the same kind of fruits or vegetables. Too many hershey's kisses will simply distract people from the breakfast display in the supermarket.\\\", \\\"If you want a more traditional looking rose bouquet, get 24 hershey's kisses; this will make a dozen roses. [substeps] For an even more rose-like bouquet, use red or pink hershey's kisses.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}