{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9df9b7cb-be0d-5767-aeb0-dd9fa9f2aaa3","task_key":"default--validation--21857","task_revision_id":"1","upstream_id":"21857","short_description":"HellaSwag validation 21857","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to relax all day (for women and girls ) [title] Gather books, magazines, mp3 players, art supplies, and nail polish. [step] Gather them legally. Do not steal these items.\",\"endings\":\"[\\\"Makeup is also optional, but it's always good to have some free. But there's one thing you can do wrong.\\\", \\\"Look around your home or buy new, it doesn't matter where from or how you gather these items the important thing to aid you to relax all day is that you gather them. Get as many of each as you think you'll need, and then a little more, just in case.\\\", \\\"The law prohibits this from being permitted in the workplace. [title] Put on a simple outfit that won't draw attention to yourself or cause a panic.\\\", \\\"Stay away from this place and don't steal anything from other people. This includes anything out of the ordinary or rare.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}