{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9cf18f06-d05c-5d04-a4dc-1f9f46e1a467","task_key":"default--validation--47269","task_revision_id":"1","upstream_id":"47269","short_description":"HellaSwag validation 47269","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to make your own crystals [title] Combine the ingredients in the container. [step] Add about a half cup (120ml) of epsom salt (magnesium sulfide) with equal parts hot water. [substeps] This will create a saturated solution.\",\"endings\":\"['The less hot your solution is, the better. Keep in mind that epsom salt floats and can discolor.', \\\"You can add later until you've applied more of the entire solution. Cast iron is the cheapest of all items on the periodic table.\\\", 'It may also improve the ph of the water. Add two tablespoons (29.5 ml) of ammonia (tsp) and vinegar (xhella vevalva) for an acidic solution.', 'A solution is considered saturated when no more solids can be dissolved into the liquid. Add a few drops of food coloring to the mixture if you want your crystals to have some color.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}