{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b1e7564a-6a38-58cb-8a0c-c3e53bcca1fe","task_key":"default--validation--15192","task_revision_id":"1","upstream_id":"15192","short_description":"HellaSwag validation 15192","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make a natural fruit perfume [title] Assemble the ingredients. [step] For this recipe, you will need the following items : 1 organic cinnamon stick , 25 organic cardamom pods , 15 organic cloves , 1 organic vanilla bean , 1 fresh organic orange peel (zest only) and 8 ounces of vodka. [substeps] Cut the vanilla bean into small pieces using a sharp knife.\",\"endings\":\"['Zest the orange peel by rubbing the peel over a cheese grater. When you reach the white portion of the rind, stop grating and move on to another portion of the peel.', 'This will make room for the cloves and the vanilla bean. You can soften the beans with tea.', 'Use the juice from 1 orange to use as a thick syrup. You may adjust the amount of vodka in the syrup a couple of weeks before making the perfume for a step-by-step decision.', 'Roll the slices into a ball using a rolling pin, then have them dip in vodka to coat them. Place the pieces in a jar with a lid, and shake hard to coat the balls.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}