{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5be04351-f82c-5f6c-963d-f759a00fc70a","task_key":"default--validation--22334","task_revision_id":"1","upstream_id":"22334","short_description":"HellaSwag validation 22334","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to clean stainless steel jewelry [title] Fill two small bowls with warm water. [step] One of the bowls will be used for washing the jewelry, while the other one will be used for rinsing it. Make sure that the bowls you are using are large enough to completely submerge the jewelry.\",\"endings\":\"['[title] Add a tablespoon of salt to each bowl. [step] As this will prevent the salt from getting into the jewelry and damaging the metal.', 'Fill both bowls with warm tap water and pour in about ¼ of a cup of warm tap water. [substeps] If you do not have a large enough bowl, you could use several smaller bowls.', '[title] Add 2 to 3 drops of mild dish soap into the first bowl. [step] If your jewelry is particularly dirty, look for a dish soap that is labeled for fighting grease.', '[title] Submerge the jewelry in the water bowl. [step] Fill each bowl half-full of warm water, and then fill each bowl with warm water.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}