{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"84868a18-6a2b-50ce-b47e-15f59f22da48","task_key":"default--validation--37889","task_revision_id":"1","upstream_id":"37889","short_description":"HellaSwag validation 37889","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make organic sunscreen [title] Fill the bottom portion of a double boiler with a few inches (10 to 20 centimeters) of water. [step] Place the double boiler on the stove and heat over medium-low heat, enough to make the water hot but not boiling. Do not attempt to mix your organic sunscreen over direct heat using a standard sauce pot.\",\"endings\":\"['[title] Place the sunscreen into the double boiler and allow it to heat. [step] When the water boils, turn the burner up to medium-high.', 'The pot of water should turn into a big ball and create a steaming motion as you heat the sunscreen. [substeps] When you buy a bottle of organic sunscreen, look for small bottles that you can fit all your ingredients and oil to.', '[title] Place the beeswax and coconut oil into the upper portion of the double boiler. [step] Slowly heat and stir the two ingredients together with a heat resistant spoon until both have melted into a liquid state and have combined.', '[substeps] The maximum amount of water can be added can be substituted for boiling water. For some instructions, please check the guidelines below for specific amounts, as well as how much water you should use.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}