{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c20b685e-4ed3-5f62-b14d-5db18c4ac8e6","task_key":"default--validation--35235","task_revision_id":"1","upstream_id":"35235","short_description":"HellaSwag validation 35235","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to make enzyme cleaner [title] Wash and chop the citrus peel. [step] Rinse the citrus peel under running water and scrub the outside with a vegetable brush to remove dirt and impurities. Pat the peels dry with a clean towel, and carefully chop the peels into half-inch (1.3-cm) cubes.\",\"endings\":\"[\\\"If you plan to use lemon juice for this cleaning, make sure to sterilize your can opener so that the juice doesn't collect on the inside. If you want to use cider vinegar instead, make a lemon juice substitute.\\\", '[substeps] Rinse the peels with running water, and place the lime juice and lime zest in a bowl. You can store the peels in the refrigerator, but make sure to use the oranges in the process.', 'The pieces have to be small enough to fit into the opening of a pop bottle. [substeps] You can use a variety or mixture of citrus peels to make your homemade enzyme cleaner, including lemon, lime, grapefruit, and orange.', \\\"Set aside once you've washed and chopped the citrus peels. [substeps] You'll know that the peels are fully ripe by the pink flesh that is visible through the peel.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}