{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c29e2489-60f9-59f4-87db-9e7524e14e06","task_key":"default--validation--28187","task_revision_id":"1","upstream_id":"28187","short_description":"HellaSwag validation 28187","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to get cat urine out of a mattress [title] Absorb excess wet urine. [step] When you discover a fresh marking that's still wet, grab a couple of old towels and press them firmly into the mattress to absorb as much of the urine as possible. Replace the towels with dry ones as needed.\",\"endings\":\"[\\\"[substeps] You can also use items that stretch or hold moisture out of an already wet mattress such as sheets or towels. Just be sure to make sure the dry ones don't soak up too much of the urine.\\\", \\\"[substeps] If you don't have pillows, you can use a pillow case instead of a pillowcase. [title] Get a bucket big enough to bring the urine to a safe, steady level.\\\", \\\"You know you've absorbed as much as possible when towels come away dry. [substeps] Launder the towels immediately to remove the urine and get rid of the smell, otherwise the cat may mark again.\\\", '[substeps] Be sure to keep the blankets or towels as absorbent and dry as possible to keep the mattress dry. [title] Pre-treat your dog with an odor-neutralizing shampoo, if needed.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}