{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e8e689b0-5c00-5880-bb7b-3719f3c80e89","task_key":"default--validation--15535","task_revision_id":"1","upstream_id":"15535","short_description":"HellaSwag validation 15535","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to live on the street [title] Keep a blanket with you. [step] Lack of sleep is a major issue for individuals living on the streets because you let down your guard when you fall asleep. Always carry your blanket so that you can take advantage of a good sleeping opportunity.\",\"endings\":\"['[substeps] Check around your home and see if you can use it as a sleeping space. It may help to sleep in a smaller room instead of a larger one.', '[substeps] Sleeping bags are warm and suitable for outdoors. Try a bivy sack, which is like a body-sized tent.', '[title] Avoid unhealthy or costly drugs. [step] Drugs can have serious effects on your personal health, including muscle and joint pain, heart health, and even cancer.', \\\"That way you don't just shuffle to your bedroom or sit at a busy spot as you sleep. [substeps] Wearing a blanket can also prevent you from accidentally shaking things up or fidgeting.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}