{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"28ddfa04-5dd2-51cf-b456-fe9e7c39150f","task_key":"default--validation--37462","task_revision_id":"1","upstream_id":"37462","short_description":"HellaSwag validation 37462","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to buy shoes [title] Buy shoes from a brand known for quality and comfort. [step] Many well-established footwear brands are known for their reliable, functional footwear. Buy a pair of shoes from one of these brands to make sure that you are getting a high quality product.\",\"endings\":\"['[substeps] Find the quality label with high quality sizes on the back of the shoes. Confirm that the manufacturer does not use excessive amounts of lotions or lotions in the shoe, as these are designed to discourage blisters and new boots from forming.', \\\"[title] Avoid pairs that are flat or flimsy. [step] Flat shoes will be easier to measure if you are buying on a flat surface or a grassy surface that isn't tall enough to stand upright.\\\", '[substeps] Buy shoes online to avoid having to buy a pair of shoes that is too high up on your feet. [title] Ask your medical or psychological care caregiver for advice.', 'Most of these companies have built their reputations on years of good customer service as well, so these shoes are likely to be covered by a generous warranty. [substeps] Find quality footwear brands by reading online reviews.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}