{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5b0d11ec-2af0-53f6-80b6-d26a338a9339","task_key":"default--validation--22499","task_revision_id":"1","upstream_id":"22499","short_description":"HellaSwag validation 22499","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to be the best maid of honor [title] Consider your role. [step] The maid of honor is supposed to be like the bride's assistant. She helps plan the wedding, makes sure that the bride has support, and tries to make the bride's job easier.\",\"endings\":\"['That means that you can perfect your duties at your wedding before you even begin planning. That means that you can perfect your skills, work hard, and make sure that the bride makes the best decision.', \\\"The bride wants to be married and the maid of honor assisting the bride should not be self-centered. [substeps] She is supposed to be able to say no, negotiate, and not be controlled by people who are desperate for the bride and groom's love.\\\", \\\"It's a big job and requires patience, time, and effort. [title] Think of ways you can be helpful to the bride.\\\", \\\"She also fits with the wedding party, and makes sure that the bride follows her day to day life. [title] Don't just be the maid of honor.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}