{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ae932b31-ca38-52aa-96dc-86615d295f0c","task_key":"default--validation--44837","task_revision_id":"1","upstream_id":"44837","short_description":"HellaSwag validation 44837","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to ride a horse on the beach [title] Find seasoned horses to ride with. [step] Give your horse a calming influence for beach riding by finding other riders who frequent the beach. If it's your horse's first or second time riding on the beach, being with other confident horses helps keep your horse calm.\",\"endings\":\"[\\\"[title] Keep at least two equine faces near you. [step] When you're riding on a beach, some horses will quietly interact with their owners.\\\", '[substeps] This is very helpful because many horses are afraid of the ocean the first time they see it. [title] Choose a time of day with few beachgoers.', \\\"[title] Loosen up your horse's relaxed neck muscles. [step] Horse's neck muscles are particularly prone to tightness from stress on the neck muscles.\\\", \\\"[substeps] Don't ask horses you haven't ridden to ride with you or you may cause them to become nervous. For the best results, point out that the whole trip is actually extremely exhilarating for the horse.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}