{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9eb595e3-9861-504c-8313-9a47d0f9b35b","task_key":"default--validation--41481--variant-17830beb5bad","task_revision_id":"1","upstream_id":"41481--variant-17830beb5bad","short_description":"HellaSwag validation 41481","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to maintain your cowboy boots [title] Spray new boots with silicone polymer spray. [step] When you purchase your boots, you need to protect them against damage from the elements. Lay out the boots on newspaper in a well-ventilated area, hold the silicone polymer spray about a foot (30. 48 cm) from the boots and spray evenly.\",\"endings\":\"['Let the product sit in the boots for around 1 to 3 hours. [substeps] If your boots are not keeping up with the amount of time required to use the spray, try going for another method or different methods.', '[substeps] If your boots are brand new, you may need to use several different sprays before you find the amount of protection you need. You may want to consider spraying yourself first with a silicone mat or roll of protective or brush cleaner.', '[substeps] You can find silicone polymer spray at department stores and some shoe stores. This type of spray is generally safe on all leather, including suede, but check the packaging to make sure.', \\\"[substeps] Too much spray can render the boots scratched, and the dust from the spray can build up on the boots' soles, making them harder to replace. [title] Wipe down the boots with a clean cloth.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}