{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d369d880-786d-5d5f-9bb1-05fa7a0193af","task_key":"default--validation--5722","task_revision_id":"1","upstream_id":"5722","short_description":"HellaSwag validation 5722","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to get your husband to listen to you [title] Choose a location where you can talk without distractions. [step] While it might be tempting to call him on his cell phone in the very moment you become upset, calling at work or while he is busy is not going to end well. [substeps] The best location is somewhere quiet, without cell phones or the tv on.\",\"endings\":\"['Lean against a wall if necessary, or lean against a wall if you can. Listen to music or a movie in another room to help you relax.', \\\"You will also be able to have a conversation without distractions. Place your husband's phone near his bed or office if you would like privacy.\\\", \\\"A flat, car can give him some privacy. Your husband may also be busy at work, so it's best to pick an indirect location.\\\", 'If you have children wait until they are asleep or out of the house to avoid any little ears. [title] Ask whether it is a good time.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}