{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3a5ce946-d26b-5816-9d04-17054547dbae","task_key":"default--validation--6561","task_revision_id":"1","upstream_id":"6561","short_description":"HellaSwag validation 6561","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to castrate a horse [title] Get your horse assessed by a veterinarian. [step] Your vet will need to assess your horse before the procedure and talk to you about the options for castration surgery. Some horses will be able to be castrated under sedation while at home in his stall, while others will need to be castrated under full anesthesia in a clinical setting.\",\"endings\":\"['[substeps] If your horse has a medical condition that does not require treatment, he may not be given castration surgery. If the hoof was cut during the procedure, this may be more than relevant to your horse.', '[substeps] For many horses, castration surgery is the safest option for a horse. If your horse is unaware of the possibility of anesthesia surgery, surgery will be the safest option for them at home.', \\\"Be aware that castration surgery can only be performed in certain conditions. [substeps] Like surgery, the process will require pressure on the horse's head and other internal organs to be numbed with gauze or a surgical bandage.\\\", '[substeps] If the horse is healthy with two scrotal testicles, then the veterinarian may opt to to sedate the horse and operate using local anesthesia. This may be done in the field or at the clinic.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}