{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"952df014-1ff7-5f91-9be0-605d743a6fd5","task_key":"default--validation--11322","task_revision_id":"1","upstream_id":"11322","short_description":"HellaSwag validation 11322","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to keep mice cool in hot weather [title] Bring your mouse indoors if it's normally left outside. [step] Some people house their smaller pets in outdoor hutches. While this may have its conveniences, it can be very dangerous in hot or humid weather.\",\"endings\":\"[\\\"[substeps] Keep in mind your pet's internal systems should be bug-proof. This will help keep your mouse away from heat sources, cooling ducts, and outside dangers.\\\", \\\"You should evacuate it from large spaces immediately if it's being abused or surroundings are very hot or humid. [substeps] If you're going to retreat a mouse on a road trip, make sure it is on a leash.\\\", \\\"Ideally, keep your mouse inside the house if it's usually kept inside. Bring it indoors only if it's going to be outside with older mice.\\\", \\\"[substeps] Mice are very susceptible to heat stress. It's partly because they're confined to a caged area (and can't retreat to cooler rooms) and partly because they have a hard time regulating their body temperatures.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}