{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"194fa709-3d7d-5f4a-8176-d202037054e3","task_key":"default--validation--45817","task_revision_id":"1","upstream_id":"45817","short_description":"HellaSwag validation 45817","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make dry shampoo [title] Obtain a small container with a perforated lid. [step] A small salt or pepper shaker is the ideal kind of container to store your dry shampoo, but you could even use a cheese shaker. [title] Clean the container.\",\"endings\":\"['[step] Use the cheese shaker to remove any dust or dirt that will float onto the container. This will also ensure that no food could get caught in the container.', '[step] Unscrew and remove the cap from the container. Use warm soapy water to wash away any impurities before you fill it with the dry shampoo ingredients.', '[step] The container may be made of plastic or glass or another material. Make sure you rinse all moisture out before using it.', \\\"[step] The container needs to be clean, including the inside and the outside of the container. You'll also want to remove and wipe down the outside to get rid of any dirt that's stuck in there.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}