{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4d9c2242-641b-5015-b9e5-88efef63f503","task_key":"default--validation--21079","task_revision_id":"1","upstream_id":"21079","short_description":"HellaSwag validation 21079","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to choose a horse trainer [title] Ask for a referral from other horse owners you know. [step] One of the best ways to get recommendations for horse trainers is by asking people you already know in the equestrian world. Ask your friends and family members or any other acquaintances who have horses what trainers they use and who they would recommend to you.\",\"endings\":\"['[substeps] Talk to trainers about different trainers you might use and what they think of the trainers and what they typically charge for their services. You want to make sure that the trainer will deal with your horse in an unbiased and honest manner, and that you trust their service.', '[substeps] If you are a member of any horse showing or sporting association, you can also ask the association for recommendation for horse trainers. You may also want to ask about horse trainers at your local feed store.', \\\"If there is a horse show you really like and it's giving you great reviews, there might be a horse trainer you would be interested in. This will give you important information about how the trainer treats and interacts with your other horses and's trainers.\\\", \\\"[substeps] Don't just request a trainer, but call the trainer as well. If you don't know much about horses, start with one good friend, relative, or family member who is familiar with horses.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}