{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c82e63c6-01a9-5eda-9175-ab9f2522b356","task_key":"default--validation--20600","task_revision_id":"1","upstream_id":"20600","short_description":"HellaSwag validation 20600","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to loosen up a stiff knife [title] Check to see what color the blade or hinges are. [step] Does the blade or hinge look slightly orange? Or even slightly powdery or white? This could be caused by the metal in the knife oxidizing to the point of the blades adhering to the casing or each other. [title] If the knife has only metal parts, simply give it a good soaking in a mineral oil.\",\"endings\":\"['[step] This can dilute the oil and help loosen up the metal on both sides. If the knife feels sticky to the touch, but not tacky, it could be stiff.', \\\"[step] Then, buff it clean in a day or two. Once you've done this, keep the knife oiled from then on.\\\", '[step] Otherwise, a mild soap that is soaked in either mineral oil or a lime helps to loosen the hardened metal layers. [title] Run the knife under very warm water.', \\\"[step] If you don't have a mineral oil, it can either be using a sponge or an alcohol-based solution. [title] Try loosening up the blade with some white vinegar.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}