{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"13f1e9a5-9a99-5182-b298-bd0b14da5ead","task_key":"default--validation--13992","task_revision_id":"1","upstream_id":"13992","short_description":"HellaSwag validation 13992","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to select a dog bed [title] Watch your dog sleep. [step] Thinking about your dog's sleeping style is an important part of selecting the right bed. Dogs that sleep stretched out completely need a big bed.\",\"endings\":\"[\\\"Look for a foam dog bed that shapes and features some padding, which can help to create a cozy, but not overly firm bed. [substeps] Look for a foam dog bed that isn't rigid, but removable, meaning that it has plenty of room, meaning that the dog doesn't move around uncomfortably.\\\", 'If your dog likes to cuddle up on itself, however, it will need a smaller bed with sides to lean on. Similarly, look at your dog to see whether it tends to be cold or hot while sleeping.', 'Your dog will not get the same amount of compression from lying on its stomach. As a result, bed dogs need the biggest, sleepiest dog bed possible in order to sleep.', \\\"Consider the size of your dog's bed: a large dog bed may be enough for your dog. If you have a small dog bed, make a smaller one.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}