{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f7fa4ce8-ca29-5177-a769-a5ad36cab7ff","task_key":"default--validation--31160","task_revision_id":"1","upstream_id":"31160","short_description":"HellaSwag validation 31160","config":"default","split":"validation","body":"{\"activity_label\":\"Work World\",\"ctx\":\"[header] How to be a farrier [title] Learn about the profession. [step] Before you decide to become a farrier, it is a good idea to learn as much as you can about the profession. Being a farrier is hard work, but it also requires a genuine interest in caring for horses.\",\"endings\":\"['The teaching is necessary to the field of farrier work, and it takes time and in order to build the skill and skill required for the job. [substeps] If you learn a lot about the field, you will be able to work into the field more readily.', 'As a farrier, you would : [substeps] Travel to your clients to change horseshoes, clean and trim hooves, and provide therapeutic shoes for lame horses. Have knowledge of horse anatomy and veterinary care, and advise horse owners if there is a problem that requires veterinary attention.', 'Gaining this ability will help you make connections and learn about animals, learn about your job, and end up going to the right locations. [substeps] Let your adviser know that the position you want to get into can provide the right opportunities and be helpful to applicants.', 'As such, the more experience you have in animal care, the more you will be able to pay attention to your work and to be knowledgeable about business. [substeps] Look for a job that is in your area that gives you plenty of experience or that is appealing to the core idea of professional riding.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}