{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4b3521fe-c8dd-519c-89b0-6acb2ce18a92","task_key":"default--validation--14156--variant-53e53e5c478d","task_revision_id":"1","upstream_id":"14156--variant-53e53e5c478d","short_description":"HellaSwag validation 14156","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to make almond paste [title] Bring water to a boil. [step] Fill a small saucepan 2/3 full with water. Warm this water on the stove on high, heating it until it reaches a rolling boil.\",\"endings\":\"['Stir the mixture continuously as it boils. [substeps] It is important to handle the water very carefully.', '[substeps] You need to blanch the almonds in order to remove the skins. Do not leave the skins on when making almond paste, since the final product will be coarser and grainier if you do.', '[substeps] Make sure the water is boiling slowly for the best results. Boiling it too quickly could cause the paste to come out of the bottom.', '[substeps] At this point, you can lower the temperature to about 32 degrees fahrenheit (degrees celsius). To accomplish this, you can add a few drops of peppermint essential oil or spearmint essential oil.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}