{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d5adb7bb-7843-5327-8553-c077bb7a6cf3","task_key":"default--validation--2725","task_revision_id":"1","upstream_id":"2725","short_description":"HellaSwag validation 2725","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to get married on maplestory [title] Be sure you want to go through with it. [step] It takes time and nx cash as well. If you've talked it over and are sure, keep reading.\",\"endings\":\"['Also, remember that only a male character can get the ring. [title] Go to moony and buy an engagement ring (you no longer have to bring items).', \\\"Having a baby together or it's for someone else will cost any money that you can go through. Not everyone you've got to marry wants to have their baby together.\\\", \\\"[title] Figure out if maplestory is right for you. [step] If it sounds like you've got the number right for you, go ahead and have a little chat with one of your friends.\\\", \\\"This article will give you a guide on how to get your marriage license before signing it. This list will help you do it more quickly, and you'll get more points by doing it later in the year.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}