{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"84d51a47-5fc6-5364-9eab-ac5da581c9d8","task_key":"default--validation--37968","task_revision_id":"1","upstream_id":"37968","short_description":"HellaSwag validation 37968","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to perform a sand ceremony [title] Select your keepsake vase or container. [step] You can use virtually any container to house your sand, though glass vases are the most common. Some options include a heart-shaped vase, cylinder jar, corked bottle, or decanter.\",\"endings\":\"['[title] Use colored sand with colors that you like. [step] Natural sand may not look like sand, and the best color combinations can be achieved by changing your final sand color.', \\\"Base your decision off of you and your spouse's tastes. Choose a stopper, like a cork or a lid, if you want to seal it.\\\", \\\"Once you've selected your container, choose a color to match the wedding theme. [title] Check the alignment of the color jar lid.\\\", 'You could also use a large bowl in which to hold your sand just for the sake of commemorating your wedding. [substeps] On rarer occasions, you may want to use something like a snail food jar or crepe container.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}