{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bff3a0cf-8633-5ece-a4b4-a38caec74af7","task_key":"default--validation--26240","task_revision_id":"1","upstream_id":"26240","short_description":"HellaSwag validation 26240","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to practice hospital visiting etiquette [title] Call first to make sure visitors are allowed. [step] Even if there are visiting hours provided on the website, etc. , the person you are wanting to see may not be well enough to see anyone at the time.\",\"endings\":\"['If that is the case, try a more distant visitor. [title] Play the \\\" white elephant \\\" game.', \\\"[title] Observe the hospital's visiting hours. [step] This is absolutely vital; non-visiting hours are used mainly for ensuring sufficient recuperation for the patients, so that they get well quickly.\\\", \\\"If you don't know someone, or if they are a stranger to you, it may be better to wait until when visitors arrive and do phone calls. However, always remember that every individual has different needs and may require special accommodations or treatment.\\\", '[substeps] If you are staying with someone for an extended period of time, it may seem rude to interrupt the person so much that she becomes agitated or starts yelling at visitors. [title] Take extra care to change their bedsheets occasionally.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}