{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e9700dbc-a776-549d-a29d-50bbde9f7d45","task_key":"default--validation--6675","task_revision_id":"1","upstream_id":"6675","short_description":"HellaSwag validation 6675","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to make fish pie [title] Cut raw boneless, skinless salmon into bite-size chunks. [step] Then cut raw undyed smoked haddock fillets with the bones and skin removed into bite-size pieces. [title] Substitute cod in place of the salmon, if you prefer.\",\"endings\":\"['[step] Cod is smooth, sweet, and very liquid, making it excellent for making sweets such as candy and cookies. As for meat, substitute beef, pork, and poultry with the smooth fish fish, to emulate the effects of hot butter and horseradish sauce.', '[step] Cod is a somewhat healthier substitution than fish. [title] Add a butter-filled pre-chopper spoon to the prepared pan or smaller pot.', '[step] For a tasty, hearty baked \\\" fish pie \\\" fresh cod and panko fish will suffice. The flesh will provide a perfect crust for the fish.', '[title] Use any white fish, such as cod, alone in the recipe for a more uniform fish flavor. [title] Add the fish pieces into the 9x13 inch baking dish.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}