{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a37d66ba-7394-5616-876c-0cc497404429","task_key":"default--validation--9989","task_revision_id":"1","upstream_id":"9989","short_description":"HellaSwag validation 9989","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to plan a cesarean section for brachycephalic dogs [title] Have your vet measure luteinizing hormone (lh) levels. [step] Timing the c-section is an important part of the planning process. Several strategies are available to determine when a c-section should take place.\",\"endings\":\"['The timing of the c-section is very important. [substeps] The vet will tell you in advance whether the dog is at least 5 pounds.', \\\"While it would be best for your dog to get tested at your vet's office, there are several methods you can use to make a rule against this. Consider : [substeps] He can confirm your dog's pregnancy through the test.\\\", 'These strategies can help your dog recover from the procedure without affecting any negative effects on its body. Your vet can also determine if you need to take your dog to a clinic or a veterinarian before the procedure.', 'One strategy is measuring lh, a hormone that stimulates ovulation (egg released from the ovary). [substeps] Peak lh levels mark the beginning of pregnancy, which lasts an average of 65 days in a dog.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}